United States Environmental Protection Agency Office of Air Quality Planning and Standards Research Triangle Park NC 27711 EPA-450/4-79-006 December 1978 Air Development of an Emission inventory duality Assurance Program ------- EPA-450/4-79-006 Development of an Emission Inventory Quality Assurance Program by PEDCo Environmental, Inc. 11499 Chester Road Cincinnati, Ohio 45246 Contract No. 68-02-2585 Assignment No. 8 PN 3327-H Prepared for U.S. ENVIRONMENTAL PROTECTION AGENCY Office of Air, Noise, and Radiation Office of Air Quality Planning and Standards Research Triangle Park, North Carolina 27711 December 1978 ------- This report is issued by the U. S. Environmental Protection Agency to report technical data of interest to a limited number of readers. Copies are available free of charge to Federal employees, current contractors and grantees, and nonprofit organizations - in limited quantities - from the Library Services Office (MD-35), Research Triangle Park, NC 27711; or, for a fee, from the National Technical Information Service, 5285 Port Royal Road, Springfield, VA 22161. This report was furnished to the Environmental Protection Agency by PEDCo Environmental, Inc., 11499 Chester Road, Cincinnati, OH, 45246 in fulfillment of Contract No. 68-02-2585. The contents of this report are reproduced herein as received from PEDCb Environmental, Inc. The opinions, findings and conclusions expressed are those of the author and not necessarily those of the Environmental Protection Agency. Mention of company or product names is not to be considered an endorsement by the Environmental Protection Agency. Publication No. EPA-450/4-79-006 n ------- TABLE OF CONTENTS Page, List of Figures iv List of Tables iv Acknowledgment v Executive Summary vi 1.0 Introduction 1 2.0 Quality Assurance Elements 3 2.1 Definition of Quality Assurance 3 2.2 Development of a Quality Assurance Program 4 3.0 Analysis of the Emission Inventory Process 6 3.1 Task Planning 6 3.2 Data Collection 11 3.3 Technical Procedures 14 3.4 Data Recording and Reporting 17 4.0 Error Potential in.the Emission Inventory Process 20 4.1 Typical Errors 20 4.2 Control Techniques 28 5.0 Implementing Quality Assurance Procedures 32 5.1 The Role of Federal Guidance in Quality Assurance Program Development and Imple mentation 33 5.2 Regulatory vs. Voluntary Approach 35 Appendix A 38 Appendix B 42 111 ------- LIST OF FIGURES No. Page 2-1 Procedures Fox Developing a Quality Assurance Program 5 3—1 Major Steps in the Emission Inventory Pro-cess 7 LIST OF TABLES No. Page 3-1 Task Planning for the Emission Inventory Process 8 3-2 Data Collection Procedures in the Emissions Inventory Process 11 3-3 Technical Procedures in the Emission Inventory Process 15 3-4 Data Recording and Reporting Procedures in the Emissions Inventory Process 17 4-1 Errors and Error Sources in the Emission Inventory Process 26 4-2 Examples of Preventive Quality Assurance Techniques. 29 4-3 Examples of Corrective Quality Assurance Techniques 30 xv ------- ACKNOWLEDGMENT This report was prepared for the U.S. Environmental Protec- tion Agency by PEDCo Environmental, Inc., Cincinnati, Ohio. David W. Armentrout was the principal author. Charles E. Zimmer was the Project Director for PEDCo. Mr. Jamee Southerland was the task officer for the U.S. Environmental Protection Agency. We wish to thank him for his guidance and professional opinions offered throughout this task. ------- EXECUTIVE SUMMARY The 1977 Amendments to the Clean Air Act include require- ments for the Prevention of Significant Deterioration (PSD), New Source Performance Standards (NSPS), emission offsets, reasonable further progress, and other considerations that could increase the number of users and the significance of emission inventory data. These considerations involve an increasing audience of private users from the industrial community in addition to the traditional users associated with control strategy development and assessment. To make the inventory data and the policies based on them more representative and equitable, to both public and private interests, quality assurance guidelines for emission inventories (similar to those designed for ambient air monitoring systems) are desirable and necessary. Quality assurance comprises three types of procedures: standard operating procedures; procedures for finding and cor- recting errors and inconsistencies; and procedures for data quality assessment (i.e. degree of confidence that users can have in the data). As applied to emission inventories, these proce- dures would focus more upon human error than upon equipment parameters. Guidelines for developing an emission inventory quality assurance program would first analyze the system to identify its components; second, estimate the potential for error and identify the errors having the greatest impact on inventory results; and third, develop techniques for the control and cor- rection of errors. The components of the emission inventory process are task planning, data collection, technical procedures, and data re- cording and reporting. Descriptions of these components follow: VI ------- 0 Task planning (a management function): identifying working group responsibilities, establishing communica- tions, developing general inventory procedures, speci- fying data format requirements, and designing data files. 0 Data collection (an operations function): identifying data requirements and sources, specifying formats, setting response schedules, and developing specific data collection mechanisms (including continuous updat- ing) . 0 Technical procedures (an operations function): all tasks related to data analysis and interpretation, and the calculation of emissions. 0 Data recording and reporting (a management and opera- tions function): determining report formats and proce- dures for coding, editing and recording. Errors include inconsistencies in the data, as well as mistakes. Typical errors in the emission inventory process can often be linked to more than one of the four components listed above. Examples of common errors are omitted or duplicated emission sources, incompatible time bases among sources, variable emission calculations resulting from the misuse of emission factors, incomplete codes, and improper or variable reporting units. To illustrate, emission sources could be duplicated in or omitted from the data base through misinterpretation of process diagrams by individuals completing permit or emission inventory data reporting forms. Entire facilities have been duplicated in emission inventory systems because of failure of agency personnel to change identification data when a facility changes names. Variable emissions estimates might appear in different invento- ries involving similar types and sizes of sources, depending on technical personnel interpretation of emission factors that are given as a range instead of an an absolute number, to use another illustration. The control techniques devised for the emission inventory process can be classified as preventive or corrective. Many of the successful control techniques are already in use in various agencies, and need only to be established on a formal basis. vii ------- Formalizing these techniques provides the basis for a visible quality assurance program. It should make the agency staff more aware of how their daily activities interface with a quality assurance program, and it will provide a reference point for management to assess the agency's needs for developing and imple- menting further control procedures. The techniques range from training programs and spot check calculations to the establish- ment of source accounting procedures and a separate coordinating function to supervise and evaluate quality assurance within the agency. Evaluation of the effectiveness of quality assurance com- bines a quantitative assessment of the integrity of the data and a qualitative assessment of the entire system. A system audit, to be performed by management, is a procedural review to deter- mine whether the various elements in the program are being prop- erly applied. Cooperation between EPA and state agencies would be essen- tial for the implementation of quality assurance procedures. The major responsibility for program development and implementation would lie at the state level, but EPA could provide substantial technical assistance based on its experience with ambient air monitoring. A pilot project could provide important cost-benefit data and demonstrate the effect of the new procedures upon agency organization. The procedures could be implemented on either a voluntary or a regulatory basis. Advantages and disadvantages are found in both possibilities. Voluntary implementation has the potential to provide better resource allocations, reduced resistance to change, and greater management commitment to the program; but it has not been entirely satisfactory when applied to ambient air monitoring systems. Regulatory implementation might be completed more quickly and have a higher degree of cost-effectiveness; but it might also place a burden on EPA, and might encounter more implementation problems at the local level. Vlll ------- SECTION 1 INTRODUCTION The number of users interested in emission inventories has been increasing since Prevention of Significant Deterioration (PSD) requirements were adopted in the 1977 Amendments to the Clean Air Act. Industries that must include emissions and source operating data with their permit applications to regula- tory agencies will use the inventories to assess the relation of PSD requirements and emission offsets to the possibilities for industrial growth. The evaluation of new energy strategies relies upon emission inventories for a measure of the air quality impact of burning various fossil fuels. Emission inventories will continue to be used in the traditional applications, such as dispersion modeling for development and assessment of air quality control strategies. Determining reasonable further progress toward attainment of national air quality standards is also a key application for emission inventory data. Economic questions and concern about energy policies can be expected to increase the use of the comprehensive emission inven- tory data. As more users realize the economic, health, and social importance of the policies that are based on the data, they will demand greater reliability. A quality assurance pro- gram applied to the production of emission inventories may im- prove the reliability of the data and the compatibility among inventories. This document reviews the emission inventory process and suggests an approach to developing a quality assurance program for it. It includes a preliminary assessment of typical errors and inconsistencies that occur in the emission inventory process, and outlines procedures for controlling or minimizing them. The concept presented here is intended to be used as a basis for 1 ------- further discussions of quality assurance in the emission inven- tory process. The assessment of inventory problems is not com- prehensive. Additional problems undoubtedly remain to be iden- tified. However, the discussion in this report does provide a preliminary indication of the nature of the problems, the impact on the data, and the feasibility of control. ------- SECTION 2 QUALITY ASSURANCE ELEMENTS 2.1 DEFINITION OF QUALITY ASSURANCE Quality assurance is the system of procedures adopted by an organization to ensure that its products or services meet speci- fied standards of quality. The procedures, which give confidence that a basic level of precision and accuracy is maintained in the product, include ones for standard operations, for finding and correcting inconsistencies and errors, and for determining product quality and reliability. A quality assurance program applied to emission inventory procedures would have'three general types of procedures. Stan- dard operating procedures would include organization planning, personnel training, project planning, and the development of step-by-step procedures for technical tasks. Techniques for finding and correcting inconsistencies and errors would include identification of potential error sources, evaluation of the impact of these sources, location of checkpoints for optimal problem detection, and a provision for timely response when problems occur. The determination of product quality and reli- ability, in the context of an emission inventory, is the same as data quality assessment. These procedures include a periodic (e.g. yearly) review of the entire inventory process, the devel- opment of standards against which to test the accuracy and pre- cision of results, and a system evaluation to maintain optimal resource efficiency. A quality assurance program for-an ambient air monitoring network is focused on physical parameters, such as pollutant analyzers, monitoring sites, and calibration equipment. The emission inventory process, on the other hand, focuses on human ------- factors (with the exception of computerized data processing). As a consequence, quality assurance for emission inventory applica- tions might be weighted more heavily toward procedures analysis, and the product quality might be more difficult to assess than in an ambient air monitoring application. 2.2 DEVELOPMENT OF A QUALITY ASSURANCE PROGRAM Within an operating system, the first steps in developing quality assurance procedures are to identify the components of the system and to assess their relationship. If structural pro- blems are identified within the organization, it is useful to reorganize or restructure the system or the emission inventory procedures to correct these problems. Potential sources of error are analyzed to determine their effect on the system products. Procedures already in use are reviewed to locate the key check- points for critical potential errors. Finally, different methods for detecting the critical errors are explored, and a combination of procedures is adopted that will detect the most errors with the least resources. Figure 2-1 summarizes this general approach to the development of a quality assurance program. In the fol- lowing sections, this approach will be applied to the emission inventory process. ------- PERFORM SYSTEM ANALYSIS IDENTIFY SYSTEM COMPONENTS AND PROCEDURES EXPLAIN THEIR FUNCTIONS DETERMINE PROCEDURAL INTERRELATIONSHIPS DOCUMENT PROCEDURES AS FUNCTIONAL GROUPS IMPLEMENT CORRECTIVE PROCEDURES FOR SYSTEM STRUCTURAL PROBLEMS I QUANTIFY ERROR POTENTIAL IDENTIFY STRUCTURAL WEAKNESSES IDENTIFY ERROR SOURCES QUANTIFY THE IMPACT OF ERRORS PERFORM SENSITIVITY ANALYSIS FLAG CRITICAL ERROR SOURCES ESTABLISH CONTROL TECHNIQUES DEVELOP ALTERNATIVE CONTROL PROCEDURES PERFORM COST-BENEFIT ANALYSIS FORMALIZE AND DOCUMENT PROCEDURES Figure 2-1. Procedures for developing a quality assurance program. ------- SECTION 3 ANALYSIS OF THE EMISSION INVENTORY PROCESS The emission inventory process can be divided into four major steps: 0 Task planning 0 Data collection 0 Technical procedures 0 Data recording and reporting This division provides a convenient system for discussing inven- tory procedures and problems, but in an actual application the categories are interdependent. As Figure 3-1 shows, the process requires input from both the management and operations levels. The sequence of steps begins with management input, progresses through tasks that are primarily performed by operations, and ends with input from both management and operations. 3.1 TASK PLANNING Task planning includes a variety of procedures that reflect agency policy and goals, and as a consequence require management decisions. The decisions address definitional, organizational, and technical problems. Table 3-1 summarizes the procedures that are included. ------- PRIMARY IHPUT GROUP MANAGEMENT OPERATIONS OPERATIONS OPERATIONS MANAGEMENT CATEGORY TASK PLANNING -*• DATA COLLECTION TECHNICAL PROCEDURES DATA RECORDING AND REPORTING Figure 3-1. Major steps in the emission inventory process ------- TABLE 3-1. TASK PLANNING FOR THE EMISSION INVENTORY PROCESS Definitional Define objectives Define requirements for meeting objectives Organizational Relate group functions Assign responsibilities Establish communications Assign priorities Budget resources Document procedures Technical Develop job descriptions Identify training requirements Develop inventory techniques Establish level-of-effort guidelines The definitional phase of the task planning step is the one in which management defines the system objectives and outlines the basis for meeting those objectives. 0 Define objectives: This procedure requires management to assess the purpose(s) of the emission inventory system. Example goals may include providing informa- tion to other agencies, providing input to control strategy testing, providing data for assessing trends, or combinations of these and other objectives. 0 Define requirements for meeting objectives: Once the major purposes of the emission inventory process have been identified, management needs to define the types of information required and to assign authority and responsibility for obtaining that information. This includes defining specific data types and formats, determining possible or necessary interfaces with other 8 ------- functional systems within the organization, establish- ing the authority to obtain and disperse the necessary emissions data, and defining data'quality goals. The organizational procedures are those that determine the total effort, the resources that will be expended, and the rela- tionship between the inventory process and other agency func- tions. 0 Relate group and agency functions; This procedure in- cludes an organizational audit to define the purpose of each group within the organization and the level of support that it can provide. 0 Assign responsibilities; This establishes the respon- sibilities and authority of the department or division for compiling and maintaining emission inventory data. It is an early step in developing a mechanism for establishing the emissions inventory. It might also include, for example, an assessment of the degrees of support that remote branch office operations can pro- vide. 0 Establish communications; This determines the flow of information between management and operations levels within each department or division of the agency. It also establishes the environment for interdepartmental transfer of technical data. 0 Assign priorities; This function follows from the assessment by the agency of the scope of its air pol- lution problems, and the degree of industrialization or urbanization of the area within its jurisdiction. 0 Budget resources; This function is the logical exten- sion of the assignment of priorities. Budget decisions reflect the expected impact of the emission inventory on agency effectiveness, and they affect the types of inventory procedures that will be adopted. ------- Document procedures; This function is the formal sum- mary of the activities mentioned above. The technical procedures help to establish the degree of confidence that can be placed in the emission inventory results. They affect the precision and accuracy of the inventory, and the degree to which results can be compared within or among agencies. These procedures are: 0 Develop job descriptions; This includes writing job descriptions for personnel who will be assigned ex- clusively to the emission inventory process. 0 Identify training requirements; This identifies the technical questions that must be addressed in compiling an emission inventory, and the functions requiring special skills or periodic updating of techniques. The training programs and schedules will be designed to aid in incorporating improved inventory techniques and to reduce confusion in performing inventory functions. 0 Develop inventory techniques: This includes decisions about the emission factors to be used, data gathering techniques, manual vs. computer data processing, tech- niques to identify sources, and workflow procedures within the organization. These are only a few examples of a range of complex functions. 0 Establish level-of-effort guidelines; This is directly related to the setting of budget and task priorities that was described under organizational considerations. It determines the kind of data collection effort to be made and the comprehensiveness of the inventory. It relies heavily on input from the definitional phase of task planning, i.e., management objectives, inventory quality goals, etc. 10 ------- The decisions made during task planning have a direct impact upon the three other steps that have been outlined for the emission inventory process: data collection, technical procedures, and data recording and reporting. 3.2 DATA COLLECTION Data collection involves significant levels of human inter- action and attention to communication techniques. The task can be divided into procedures for compiling a new inventory, and those for the routine maintenance of an existing one. Table 3-2 summarizes the major procedures for each case, and brief descrip- tions follow. The procedures listed for compiling a new inven- tory have many characteristics that easily could causs these procedures to be characterized in the task planning phase of the inventory process. They are listed in the data collection phase, however, because they are flexible depending on specific project requirements. The task planning procedures previously identi- fied, however, provide more of an organizational basis for the entire emission inventory program than for any specific applica- tions that might occur within the program. TABLE 3-2. DATA COLLECTION PROCEDURES IN THE EMISSIONS INVENTORY PROCESS Compiling an inventory Identify data requirements Identify sources and availability of data Determine reliability Specify formats Determine time constraints Determine the mechanics Maintaining an inventory Estimate resource requirements Establish a schedule Determine the mechanics The procedures for compiling an inventory are: 11 ------- Identify data requirements: This function (which is directly affected by the priority and budgetary con- straints imposed by the task planning phase) determines whether primary or secondary data sources are to be used, the expected number and types of emission sources to be inventoried, and the specific data elements desired. Identify sources and availability of data: The range of data sources will influence the decision to use primary or secondary data. Primary data sources in- clude industrial or commercial facilities, and sec- ondary sources include published data. Primary sources could also include other data bases, depending on how they were compiled. Data availability depends on state and local regulations, confidentiality constraints, and limitations of time and money. In the case of primary data, availability can be affected by the political atmosphere between agencies and the managers of a facility. Determine reliability: The professional capabilities and attitudes of the agencies or facilities supplying either primary or secondary data must be judged. Their data collection techniques must be assessed, as well as the purpose for which the data have been compiled. Specify data formats; The formats for the inventory will depend on its purpose and scope, and on the mode to be used to store and report the data. If the data are collected as part of a formal regulatory reporting requirement, they will have to conform to a specified format. Formats of data that are acquired must be reviewed to determine the cost and time needed to make conversions. One of the most important elements in selecting a format is units of measurement. 12 ------- 0 Determine time constraints; These will usually be fixed by a reporting deadline from a regulatory agency. The lead time needed to collect primary data may not be compatible with the stated deadlines. In these cases it may be necessary to choose between the quality or completeness of the data, and project budget constraints. If the constraints are rigid, the scope of the project may need to be changed. 0 Determine the mechanics: This includes designing the questionnaire, data recording forms, and letters of request; and establishing procedures for tracking data through the emission inventory process. These func- tions are limited by the organizational decisions made during the task planning phase. Each of these considerations affects the completeness of the inventory and the quality of the data. The data collection procedures described above also apply to the maintenance of an existing inventory. In addition, there are some procedures specific to inventory maintenance that will affect the suitability of the inventory for future uses. Three major ones are described below: 0 Estimate resource requirements: These requirements depend on the comprehensiveness of the inventory to be maintained and on the schedule for updating it. Esti- mates are based on data volume, frequency of updates, mode of update (manual or computer), and number of personnel to be assigned. Each of these parameters is established during the task planning phase. 0 Establish a schedule: Scheduling relies on the re- source requirements that are budgeted during the task planning phase. Update frequency affects the way the data is maintained in the inventory file, and it also influences the resource requirements for specific 13 ------- applications. A major problem with many data bases is that the sources use different time periods. This variation can reduce the reliability of the data for strategy testing or other applications. 0 Determine the mechanics: Areas of concern in this category include system compatibility (e.g., permit systems vs. emission inventory systems), resource availability, and interagency and intraagency com- munications. The update procedures should strive for a common time base among emissions sources. The update should be automatic, and should include editing and file checking procedures. The mechanism must include a method to identify emission sources that have undergone significant changes, i.e., addition, deletion, or changes in physical or operating parameters since the last update. 3.3 TECHNICAL PROCEDURES The third task category in the emission inventory process comprises the technical procedures. These procedures (see Table 3-3) include data analysis, emissions calculations, and some important data flow and control techniques that directly affect the use of the results. The technical procedures, which can be divided into technical and clerical functions, are the ones most often discussed in documents dealing with emission inventories; and they have been the subject of numerous seminars and training sessions. This voluminous information about technical procedures, however, does not make them any more important in terms of their impact on inventory quality and data applications than the other three categories. These technical procedures are part of a con- tinuing effort to assess and update emission inventory data as well as the daily activities that are used to collect and report the data. The technical portion of this category involves decisions about data input to the inventory process and the 14 ------- TABLE 3-3. TECHNICAL PROCEDURES IN THE EMISSION INVENTORY PROCESS Technical Conduct training activities Determine applicable emission factors Prepare support materials Calculate emissions Review results Clerical Assemble reporting and calculation forms Develop system to assign source identification numbers Control work flow preparation and review of emissions estimates. Descriptions of these functions are given below: 0 Conduct training activites; This function requires the participation of both management and technical staff. Training activities include both periodic, e.g. semi- annual or annual, seminars to update procedures and intermittent special sessions relating to specific problems. Special sessions might be held, for example, to discuss requirements for a special inventory or update that may differ in some respects (data formats, detail required, etc.) from the routine agency inven- tory procedures. Periodic training sessions might include discussions of new emission factors, procedural changes to eliminate specific operating problems or to interface with a new data reporting system, etc. 0 Determine applicable emission factors; This is a con- tinuing responsibility that should be assigned directly to a member of the emission inventory staff. Duties include staying informed about changes in emission \ 15 ------- factor technology, and making decisions about when an agency should develop and apply its own emission factors, 0 Prepare support materials: These materials are the illustrative calculations, tables of codes, and appro- priate references for the technical staff members who will work on the inventory. Senior technical staff act as advisors to junior staff who are preparing the emissions estimates, but some questions can easily be answered without consulting the senior staff. Support materials also help to maintain consistency within the inventory. This function is closely related to the training activities function. Support materials change as inventory data gathering, emission estimating, and data reporting techniques change within the system. 0 Calculate emissions: The success of this critical function has a direct relation to the amount of prep- aration that has been made, through the three functions described above. 0 Review results: This function entails spot checks of calculations; of assumptions and engineering judgments; unit conversions; and adherence to established proce- dures. To accomplish this review, project or group managers need to set guidelines about documenting tlhe assumptions and calculations that are made. The fre- quency of these checks will depend on the staff exper- tise, size of the inventory, etc. The clerical portion of these procedures comprises the following recordkeeping functions: 0 Assemble reporting and calculation forms: This in- cludes a review of the format requirements for the data base to which the data will be added. The format for recording emissions estimates and associated inventory data must be consistent for all emission sources. 16 ------- 0 Develop a system to assign source identification numbers: The initial list of sources to be included in the inventory must be reviewed to see if it contains duplicate sources with different identification numbers, especially sources that have had name changes. A control list of assigned identification numbers and their associated sources is established to avoid dupli- cate number assignments. If data are taken from several data bases, a cross-reference must be estab- lished to prevent double counting of sources. These cross-reference lists may also help to flag question- able cases, such as sources that have had name changes or that should be deleted from one of the input files because they are no longer operating. 0 Controlling work flow: This is a technical planning procedure that routes the work through the data han- dling and review process and maintains an inventory of work that is in process or is complete. 3.4 DATA RECORDING AND REPORTING The procedures for data recording and reporting, summarized in Table 3-4, apply to both manual and computer data handling techniques. TABLE 3-4. DATA RECORDING AND REPORTING PROCEDURES IN THE EMISSIONS INVENTORY PROCESS Determine format requirements Develop coding and recording guidelines Implement editing procedures Implement data verification procedures In an actual application, many of these procedures become part of the technical procedures. They are discussed separately here because many are closely related tasks, easily analyzed and 17 ------- controlled within the context of developing and applying quality assurance procedures. The data recording, and- reporting functions are described as follows: 0 Determine format requirements; This requires a review of the system to which the inventory results are to be added. Important considerations include the source identification system, units of measurement, special codes (such as control and process identification codes), appropriate field sizes, and implied or ex- plicit decimal requirements. The final reporting de- sign is dictated by the data base format review. 0 Develop coding and recording guidelines; This follows directly from the development of format requirements. For computerized files, it includes specifications for field adjustment, embedded blanks, decimals, alphabetic and numeric fields, and definition of terms referenced on the coding forms. A familiar example is the U.S. EPA guidelines for the National Emissions Data System (NEDS). The NEDS system, the repository for emissions data reported to EPA in compliance with the Clean Air Act, is extensively documented. Seminars and training courses have been conducted to teach data contributors at the agency and contractor level about the specific coding requirements of this system. Similar specifi- cations can be developed for any system. 0 Implement editing procedures: A series of quality checks must be established to ensure that units specifi- cations and coding and reporting requirements have been met. Effective editing procedures also include checks for missing data, improper codes, and similar problems. The NEDS editing procedures can serve as a model for agencies developing their own systems. The editing procedures provide a degree of input to assessing 18 ------- system efficiency. Editing statistics regarding number and types of data recording problems can provide feed- back to management to use in the continuing training activity cycle. Implement data verification procedures; Routine proce- dures must be established to verify the validity of data submitted to the inventory system, to correct errors flagged during data review and editing, and to obtain data elements that are missing from the file. As an example, a state agency might routinely (e.g. semiannually) summarize missing or questionable data elements and request the responsible facilities to respond by supplying new or corrected data. In addi- tion, it might be useful for an agency to recap the facilities, sources, and key data parameters repre- sented in the inventory file and ask the data origi- nators to briefly review and reconfirm or change the data if necessary. This type of verification procedure could instill more confidence on the part of data users in the inventory file, result in a better working rela- tionship between agencies and facilities in their jurisdiction, and result in more current data than might be available if the inventory relied on informa- tion directly from a permitting system, which typically might be on a 3 to 5 year cycle. Appendix B provides an overview of the types of editing and verification procedures used at NADB. These procedures can serve as a model for this phase of an emission inventory quality assurance program. 19 ------- SECTION 4 ERROR POTENTIAL IN THE EMISSION INVENTORY PROCESS An assessment of the potential for error in emission inven- tories provides a clear demonstration of the need for quality assurance procedures. Each category that was outlined in the analysis in the preceding section has an impact upon the total potential for error. The task in this section is to identify the types of errors and their sources, and to estimate their im- portance in affecting inventory reliability. In the context of this discussion, the term "error" means not only wrong answers (e.g. incorrect calculations), but also inconsistencies among emission inventory results (e.g. cutoffs for facility size). Both problems are equally important, because they can lead inven- tory data users to wrong conclusions, and can prevent the results from being useful for comparisons within or among states or regions for the purpose of assessing control programs. Most of the problems that occur during the inventory process can be directly related to one or more of the four task cate- gories: task planning, data collection, technical procedures, and data recording and reporting. These relationships become especially important for establishing the controls that might be used in a quality assurance program. 4.1 TYPICAL ERRORS The errors and inconsistencies that are discussed here are among the most obvious that can happen during the inventory process. Five errors have been chosen for discussion. The list is not exhaustive, however: it is only meant to illustrate the types of problems that can occur, their impact, and control techniques that can be applied in a quality assurance program. 20 ------- The causes of the errors are poor planning, constraints in the state of the art of inventory techniques, and random human error. Each error will require a different control. The development of a quality assurance program for emission inventories will require an analysis similar to but more comprehensive than is possible here, including sensitivity testing where applicable. 4.1.1 Missing or Duplicate Emission Sources It is possible for sources to be inadvertently missed or double counted in the emission inventory. This error would not be expected to be critical for smaller sources, but could be so for large ones (depending on land use patterns). The problem could result from poor bookkeeping or inadequate procedures for reviewing data from major facilities. Quantifying the error potential for this problem may only be useful in a subjective context. If emissions sources are missing from the inventory, the cost could be the adoption of relaxed controls that could have a negative impact on local health and welfare. If emissions sources are duplicated, the cost could be the adoption of controls that are too strict and that could restrain industrial competition or retard area growth. 4.1.2 Errors in Locating Sources The location of a facility or emission source can be referred to by address, by UTM coordinates, or by latitude-longitude coordinates. Errors in coding the location could have a sig- nificant effect on dispersion modeling results, and could intro- duce bias into the evaluation of control strategies. The error could result from misreading maps, from reporting the coordinates of an administrative office instead of the emission source, or from transposing digits during data recording. An-example of an extreme error in recording UTM coordinates was discovered during a dispersion modeling application. The data had been taken directly from a state agency emission 21 ------- inventory file. For one TSP source (25 tons emitted per year), the coordinates were recorded as 320.7, 596.4; they should have been 320'.'f, 4596.4. In other words, the emission source was represented in the file as being 4000 kilometers away from its real location. Admittedly, this was a gross error that probably resulted from improper coding of a computer load sheet or a keypunch error. The impact on modeling results of this type of error would depend both on its magnitude and on the size of the associated point source. The cost of an error in coding a location might include the time required to trace the problem, additional computer runs during the modeling process, impact on project schedules, and the time for checking other coordinates in the file. In the example given, other coordinate errors were found that were not as great. Errors of this type, however, were numerous enough to cause the user to doubt the validity of all the coordinates in the file. This problem could have been avoided and the potential costs reduced if routine procedures had been introduced into the in- ventory process to verify this critical data element. 4.1.3 Divergent Time Frames Most emission inventory systems are linked in some way to operating permit systems. Operating permits are generally issued for more than one year, and the year of maturity depends on the year of issue. As a consequence, the data in the permit system and in the inventory system that uses it represent different years for different sources at any one time. In other words, the. inventory update procedure never allows the system to catch up with itself so that all emission sources can be represented for the same time period. The problem may be compounded if a sizable lag exists between the times the permit system data and .inventory file are updated. In these circumstances, data in the inventory file may not represent current source conditions, especially if the source technology has changed or if economic and business 22 ------- cycles have forced major changes in operating schedules since the time the permit was issued. One county chosen at random from an emission inventory file showed the following percentages of total TSP potential emissions for each year of record: 1978 0 percent 1977 67 percent 1976 13 percent 1975 10 percent 1974 '10 percent If modeling were performed to represent the year 1977, at least 33 percent of the data in the file would first need to be updated. If modeling were desired for years in the past, the effort would depend on whether the agency maintained historic records. The problem of variable time frames could be alleviated, but not entirely solved, by verifying or adjusting the data in interim years. To reduce the cost to both industry and agency, the adjustment could be operated on an exception approach; that is, plant management would be asked to comment only on changes that had occurred since the permit was issued. The cost of this problem would be expressed in terms of the resources needed to update the inventory as a supplement to the normal permit procedures. The frequency with which data from the same year are needed would influence the decision to perform up- dates. Costs could well outweigh benefits, in this as in other applications. 4.1.4 Emission Factor Reliability Depending on how they are derived, emission factors vary considerably in their precision and accuracy. The EPA has assigned reliability ratings to its published emission factors, with a low rating denoting a factor derived from information too poor to allow its use with a high degree of confidence. In this 23 ------- case, an agency might justifiably decide to develop its own factor. Because this would be a subjective judgment, it would best be handled by a joint agreement between EPA and the agency. Some of the factors that have been published show a range, and an industry or an agency could select either end of the range to prepare its emissions estimates. Although little evidence could be given to support either value, the difference in esti- mates would be significant. For example, the range of hydro- carbon emissions from the combustion of wood and bark wastes in * boilers is listed in Supplement 7 of AP-42 as 1 to 35 kg/MT (2 to 70 Ib/ton). A footnote instructs users to apply the lower value for boilers that are well designed and operated, but an inept technician could fail to read the footnote and decide to apply the average at 17 kg/MT. These problems with emission factors might be alleviated by increased efforts at EPA to improve questionable factors, or by agency decisions to develop their own factors in such cases. Either approach is costly and time consuming. The EPA is con- tinually involved with the development and analysis of emission factors, and an expanded effort may not be cost-effective. An in-depth study would be needed before proposing such a solution. Agencies concerned with quality assurance in the emission in- ventory process should carefully review the effect of emission factors on agency policy and, ultimately, on industry and the economy. 4.1.5 Calculation Errors Errors in calculations can occur as a result of transposing digits, entering the wrong numbers in a calculator, or misinter- preting either the emission factors or the instructions for their application. The sulfur dioxide emission factor for oil combus- tion in a power plant illustrates the typical range of error. * Compilation of Air Pollutant Emission Factors. 2nd edition. U.S. Environmental Protection Agency. AP-42, February 1976. 24 ------- The factor is 19 kg/10 liters of oil burned. The number could conceivably be entered as 91, which would increase the estimated emissions by a factor of 4.8. This kind of error is random, and the only preventive measure available is to emphasize to staff members the importance of being careful in their calculations. Calculation errors could be detected through a spot check, or through a systematic review of emissions vs. throughput. The latter technique, which could be either manual or computerized, would detect gross discrepencies. 4.1.6 Characterizing Errors The errors discussed above were chosen because they repre- sent a variety of problems that have an impact on an emission inventory, and have a variety of causes and potential solutions. The impact of errors on inventory results can best be assessed through a comprehensive sensitivity analysis of each type of error. This major task would require substantially more re- sources than are available for this report. It is possible here, however, to characterize the potential causes of the common errors and to suggest techniques for their control. This characterization can provide a basis for a more thorough analysis that might be performed in the future. Table 4-1 summarizes several types of errors that can occur in the emission inventory process. This summary also shows the potential causes of these errors and the procedural category to which each is related. Although only 10 error types are listed in the table, a total of 34 potential reasons for these errors have been suggested. This indicates that multiple problems in the inventory process, often related to more than one procedural category, can result in the same types of errors in the inven- tory. The error sources, then, rather than the errors them- selves, are the key for developing control techniques. 25 ------- TABLE 4-1. ERRORS AND ERROR SOURCES IN THE EMISSION INVENTORY PROCESS Error Potential sources Related procedure category (Ti Missing facilities or sources Duplicate facilities or sources Missing operating or technical data Erroneous technical data Improper facility location data Permit and inventory systems out of phase; errors in estimating potential emissions; lost paper- work; problems with computer file updates Name changes through corporate acquisitions; use of multiple data sources with different source numbering schemes Ambiguous data request forms; intentional deletion by facility staff; inadequate followup pro- cedures; inadequate project control, i.e., no tentative indi- cation of inventory size Misinterpretation of data request instructions; assumed units, faulty conversions, etc.; inten- tional misrepresentation by the facility; poor handwriting Recording coordinates of facility headquarters instead of the operating facility; inability of technicians to read maps; changes in UTM zones All categories Data collection Data collection, task planning Data collection Data collection, tech- nical procedures (continued) ------- TABLE 4-1 (continued) Error Potential sources Related procedure category Inconsistent area source categories or point source sizes Inaccurate or outdated data Errors in calculations Errors in emission esti- mates Reported emissions wrong by orders of magnitude Failure to designate inventory cut- offs Mixed use of primary and secondary data without a stan- dard policy Transposition of digits; decimal errors; entering wrong numbers on a calculator; misinterpreting emission factor applications Imprecise emission factors; applying the wrong emission factor; errors in throughput estimates; improper interpretation of com- bined sources; errors in unit con- versions; faulty assumptions about control device efficiency; ranges of sulfur/ash contents in fuels Recording the wrong SCC code for subsequent computer emission calculations; ignoring implied decimals.on computer coding sheets; transposition errors; data coding field adjustment Task planning Task planning Technical procedures Technical procedures Data recording and reporting ------- 4.2 CONTROL TECHNIQUES The development of an effective quality assurance program rests upon a sound assessment of system procedures and identifi- cation of potential sources of error. Techniques for controlling errors are as important as those for detecting them. Techniques for error control can be classified as preventive or corrective. Preventive techniques are those that are applied as part of the task planning procedures. Classroom instruction, for example, is a preventive technique, since it flags typical problems for those participating in the inventory process and offers ways to avoid them. Table 4-2 lists examples of preven- tive quality assurance techniques. These techniques are listed as they might apply to the four major categories of emission inventory procedures. The types of problems shown in the table are those that might be prevented with each technique. Many of these techniques are already in use, but they have not been formally established. In fact, several were previously listed as part of the normal operating procedures in the emission inventory process. This suggests that many of the control techniques might be implemented with few changes in normal agency procedures. Corrective control techniques are those that are implemented to detect errors that cannot be controlled by the preventive measures. These techniques, some of which are listed in Table 4-3, reduce random errors such as digit transposition. Their primary purpose, however, is to detect deficiencies in the emis- sion inventory process. The techniques can provide useful feed- back to the task planning stage and can provide a basis for improving the preventive control techniques. Spot checks of coding forms or calculations is an example of a corrective con- trol technique. When developing the emission inventory quality assurance system, the control techniques related to the most critical types of errors must be given priority. 28 ------- TABLE 4-2. EXAMPLES OF PREVENTIVE QUALITY ASSURANCE TECHNIQUES Procedure category ro vo Task planning Data collection Technical procedures Data recording and re- porting Control techniques Training programs, certifica- tion Establish position for a quality assurance (QA) coordinator Edit data request forms Data review by local agencies Visits to largest polluters List current source ID Orientation sessions Develop reference materials Assign technical staff specifically for codinq data Problems controlled Misapplication of emission factors; incorrect units and codes; misinterpretation of source data Outdated inventory techniques; inappropriate emission factors; ineffective control techniques; overcontrol in the QA program; QA program not in phase with inventory procedures Burdensome forms resulting in data collection delays or non- compliance; inconsistent reporting units, etc. Erroneous source data reported, missing sources Missing sources, significant volume of county emissions un- confirmed Duplicate or missing sources Misunderstandings in approach or systematic errors found with corrective controls Unfamiliarity with processes, emission factors, codes, units, etc. Inconsistent coding, field adjustments, etc. ------- TABLE 4-3. EXAMPLES OF CORRECTIVE QUALITY ASSURANCE TECHNIQUES Procedure category Task planning Data collection Id O Technical procedures Data recording and report ing Control techniques Annual system audits Local agency review of com- pleted inventory data Review years of record at local agency level Engineering review of major facilities Spot check calculations Check total facility or emissions against historical inventory data Edit computer data Problems controlled QA procedures not costeffec- tive, QA procedures inappro- priate for updated inventory techniques Missing or duplicate sources, inappropriate source controls, improperly combined sources Inconsistent time representa- tion among sources. Interim operating changes not re- flected in permit or inven- tory files Data or process misinterpreta- tion Improper reporting units, random calculation errors, systematic calculation errors related to inadequate training or information Inconsistent inventory tech- niques or data sources Coding errors, e.g., improper source codes, unreasonable emissions vs. throughput, field adjustment, embedded blanks ------- The system audit is a control technique that is principally a management task. It entails a periodic, complete review of the quality assurance system. The auditor looks at current proce- dures and their effectiveness, assesses the adequacy of inventory procedures to maintain state-of-the-art technology, and takes steps to see that the quality assurance system changes according- ly. The emission inventory system should include a mechanism for user feedback. Both data users and data contributors must be able to communicate with management regarding problems they are having applying or preparing data. Information regarding data preparation problems that occur at the data source could result in revised data format or reporting requirements that could expedite data preparation. Information regarding the applica- bility of data for specific uses could result in improved emis- sion factors, improved data summary formats, retrieval options, etc. 31 ------- SECTION 5 IMPLEMENTING QUALITY ASSURANCE PROCEDURES Developing a program and publishing guidelines are only the first steps in implementing an effective quality assurance pro- gram for emission inventories. Informing agencies of the advan- tages of the program, and assisting them in adapting to it, are equally important. The examples presented in other sections of this document provide a starting point for that orientation. Even if agencies are convinced that quality assurance programs are important, two significant problems could impair successful implementation: resistance to organizational change, and budget constraints. These problems are encountered during any major procedural change, and are not unique to quality assurance. A pilot project might reduce the resistance that stems from the pertreived "impact of quality assurance procedures on the routine operation of the agency. The EPA would•develop a quality assurance program and implement it in a typical agency, an ap- proach having two important advantages. The first is the mech- anism the pilot project would provide for debugging the system. It is much easier to develop procedures on paper than to imple- ment them to the satisfaction of the users. Experience in implementing quality assurance programs for ambient air moni- toring, for example, shows that one format does not apply to all organizations. Changes in report forms and in the sequence or timing of procedures must be flexible enough to allow the program to be absorbed into the total monitoring procedures, rather than altering the total monitoring structure to conform to the quality assurance program. Practical flexibility should be a recognized requirement in a quality assurance system for emission inventories. 32 ------- The second advantage the pilot project offers is the real cost data it could yield. The potential drain on agency re- sources is an objection often made to adopting quality assurance procedures in ambient air monitoring or laboratory systems. A pilot project could provide a baseline for estimating the cost of various tasks in the program, and it could provide empirical data to support cost-benefit analyses. Because of the health, welfare, and economic implications of the emission inventory data, cost must be discussed in relation to the benefits of a quality assur- ance program: it is not reasonable to preclude better controls based on cost alone. A pilot project, in conjunction with a cost-benefit analysis, might sort out the essential elements of a quality assurance program from those that can be implemented gradually. 5.1 THE ROLE OF FEDERAL GUIDANCE.IN QUALITY ASSURANCE DEVELOP- MENT AND IMPLEMENTATION The EPA is in the best position to develop a basic quality assurance program for emission inventories. Leadership by EPA could prevent duplication of effort by state and local agencies, and could provide an opportunity for those agencies to apply the resources that would normally be used for program development toward implementation and operation. In ambient air monitoring systems, the implementation phase has been shown to take a large share of the resources for the quality assurance program; and the same situation will probably apply to the programs for emission inventories. The implementation phase would be the major respon- sibility of the state agencies. Besides avoiding duplication of effort, the role of EPA in program development would provide the consistency needed for a basic level of confidence in the emission inventories compiled by different agencies. The minimum standards for quality assurance would include, conceptually, those elements identified in the system and sensitivity analyses as being most critical to the accuracy and precision of inventory data. The size and types of 33 ------- sources, frequency of update, specifications of the data sources, data validation procedures, procedures for developing non-EPA emission factors, and similar considerations would be covered by the basic program. Other elements—the format for internal agency procedures, reporting formats, organization structure-- might be suggested, but the agencies would be able to tailor them to their own organizational constraints. Implementation of ele- ments beyond the base level would be optional, and would repre- sent a fine tuning of the quality assurance program. The specific responsibilities of the EPA in developing the basic program would include the following: 0 Perform a complete system analysis: This analysis of the emission inventory process would be similar to, but more comprehensive than, the analysis in this document. It would include a complete list of the errors and inconsistencies in an inventory as well as a sensi- tivity analysis of each error type. 0 Develop step-by-step procedures: This task would include an outline of the inventory procedures and identification of the specific quality checks asso- ciated with each. Instructions would be given for decisions On frequency of quality checks and procedures for error correction. The step-by-step procedures would be developed for each of the specific procedural categories of the inventory process: task planning, data collection, technical procedures, and data re- cording and reporting. 0 Develop guidelines for resource requirements: The guidelines would provide planning support for agency management during the implementation phase. These three basic tasks would lay the groundwork for a basic program in any agency, even though the specific details of each program may vary according to organizational activities and resources. Appendix A is an example of .an outline of a quality assurance program, which can be used as a starting point or checklist for the development and implementation of a prototype•. program. 34 ------- 5.2 REGULATORY VERSUS VOLUNTARY APPROACH Quality assurance programs for emission inventories can be implemented on a national basis either voluntarily or by regula- tion. Experience with quality assurance programs in ambient air monitoring systems can provide insight about the best approach to use. Guidelines for quality assurance in ambient air monitoring systems have been available through the EPA for 4 to 5 years. These guidelines are pollutant-specific, and this characteristic could probably be cited as contributing to delays in implementing them among various states. In addition to the pollutant-specific guidelines, a general guideline to be used by agencies as a reference for developing and implementing quality assurance procedures was published in 1975, after the initial v;rsions of the pollutant-specific guidelines had been issued. The overview was a discussion of the elements of a quality assurance program, but it did not include specific procedures for development and implementation. It is possible that specific implementation procedures may have speeded the implementation process among the states. For whatever reasons—problems in conceptualizing the ap- proach to implementation, budget constraints, organizational resistance to change, differing agency priorities — the states adopted the procedures at various rates. Some states have ap- parently still not documented formal quality assurance programs for ambient air monitoring. The EPA used the voluntary approach for establishing quality assurance in ambient air monitoring. Implementation lags among states have prevented the direct comparison of data across geographic areas, and have caused EPA to be concerned about the value of selected air quality data. The concern about data quality has resulted in the inclusion of mandatory quality 35 ------- assurance procedures in the proposed regulatory revisions for air * quality surveillance and data reporting. Tfre 'voluntary appTreraefr feo> implementation- has potential advantages: 0 The opportunity for phased implementation by allowing agencies to implement nonessential but desired elements as resources allow. 0 Reduced resistance to change an'd- improved cooperation by fostering the feeling of mutual agreement and realization of mutual benefits. 0 Greater management commitment through agreement rather than compliance. Potential disadvantages include: 0 Inconsistencies in procedures and data quality result- ing from different agency priorities. 0 . Duplication of efforts in developing the basic program, especially if state agencies elect to replace the recommended EPA guidelines with their own. 0 Lack of control of basic program elements by EPA. 0 Variable schedules for implementation depending on resource availability and agency interests. If EPA decides to adopt the voluntary approach, a critical review of the ambient air monitoring experience would be useful to identify problems and to prevent them from occurring in the emission inventories. The regulatory approach has the following potential advan- tages: 0 More rapid implementation and, consequently, an earlier assurance of data quality for nationwide comparisons. 0 A logical tradeoff in the regulations, by reducing the semiannual reporting requirement to an annual reporting requirement (as mentioned, in the proposed regulatory revisions). "EPA's Proposed Regulatory Revisions for Air Quality Surveil- lance Data Reporting," 43 CFR 34892, August 7, 1978. 36 ------- 0 Cost savings in PSD and strategy testing and related applications by eliminating the need for special pur- pose inventory updates. Potential disadvantages include: 0 An increased burden on the EPA to provide timely guide- lines and implementation assistance. 0 The possibility of developing programs that are less than adequate because of shortened review procedures or inadequate planning. 0 Organizational resistance to regulatory approaches as they impinge on state agency autonomy. If EPA decides to adopt the regulatory approach, the re- quirements must be cost-effective and flexible enough not to place undue strain on state agencies that might have irore sig- nificant problems to contend with than their emission inven- tories. The most utility with the fewest implementation problems might be realized through the voluntary approach. Maximum cost- effectiveness, on the other hand, might be realized through the regulatory approach. The cost-effectiveness would come primarily from eliminating the need for agencies or industries to conduct separate inventories, either point source or area source. More- over, the increased accuracy and precision of the data, and the consistency of the time base represented in the inventory, should provide greater reliability in the decisions made with the in- ventory data. These decisions and environmental policies have a significant effect on the health, welfare, and economy of the region where they are applied. 37 ------- APPENDIX A EXAMPLE OF A FORMAT FOR AN EMISSION INVENTORY QUALITY ASSURANCE PROGRAM 38 ------- 1.0 Quality Assurance Policy Statement 1.1 Purpose of the program 1.2 Scope 2.0 Summary 2.1 Organization chart 2.2 Emission inventory tasks and assignments of responsibility 2.3 Information flow 2.4 Summary of control techniques and relation to informa- tion flow 2.5 Audit procedures 3.0 Technical 3.1 Task Planning 3.1.1 Training and staff qualification 3.1.2 Schedule and frequency of updates 3.1.3 Quality assurance coordinator - duties and responsibilities 3.1.4 Data sources 3.2 Data collection 3.2.1 Forms and procedures 3.2.2 Data review 3.2.3 Quality assurance controls 3.3 Technical procedures 3.3.1 Emission factors 3.3.2 Data flow 3.3.3 Handling questions and problems 3.3.4 Review procedures 3.4 Data recording and reporting 3.4.1 Recording and coding forms 3.4.2 Rules for data coding 3.4.3 Data editing procedures 4.0 System Audits 4.1 Audit responsibility and schedules 4.2 Procedures 4.2.1 Elements 4.2.2 Schedule 4.2.3 Audit report 39 ------- 1.0 QUALITY ASSURANCE POLICY STATEMENT The policy statement is the formal declaration of agency commitment to developing and maintaining a quality assurance pro- gram. The policy statement defines quality assurance as related to agency activities, explains the need for the program, and provides the basis for committing resources to the program. 2.0 SUMMARY The quality assurance program summary is an executive sum- mary of the technical procedures. It provides an overview of the major components of the program without explaining the details of daily operating procedures. The summary explains the flow of emission inventory data through the agency, shows graphically the interaction among functional groups, and flags the points in the inventory procedures where quality assurance techniques are applied. It lists the control techniques and the frequency of their application. All quality assurance reporting forms, data collection forms, computerized data coding forms, etc., are included in the executive summary. 3.0 TECHNICAL The technical procedures include the operating details of each of the four major emission inventory task categories. This segment of the quality assurance program documentation must include step-by-step procedures for routine operations as well as details of checking procedures. The section on emission factors, for example, must include all current factors to be applied and must document the source of each. The documentation of these, as well as all other elements of the program, should be in looseleaf notebook form to allow the documentation to be changed easily>as procedures are changed. This is an important point, because an effective quality assurance plan is flexible and subject to change. 40 ------- 4.0 SYSTEM AUDITS This segment of the plan outlines the procedures for manage- ment to follow in reviewing the effectiveness of the program. The documentation specifies the records and procedures that will be reviewed and the frequency of review. It provides guidelines for management to use in assessing the program, and it specifies a management report format that can be used to compare audit results over time. The system audit is important, because it can flag procedures that are not cost-effective, or, on the other hand, it can indicate the need for more procedures to improve data quality. 41 ------- APPENDIX B OVERVIEW DESCRIPTION OF NEDS QUALITY ASSURANCE PROCEDURES 42 ------- The National Emissions Data System (NEDS), maintained by EPA's National Air Data Branch (NADB), incorporates several features of an emission inventory quality assurance program. These program elements are concerned with editing and validating data that have been reported to EPA from state air pollution control agencies. Although these procedures do not constitute a total emission inventory quality assurance program, they provide a good model for developing the data reporting portion of such a program. Figure B-l illustrates the data flow through the data auditing procedure that has been adopted for NEDS. Note that the system provides for continual communications among data origina- tors (state agencies), data collectors (EPA regions and NADB), and data users (industry, government agencies, etc). The important features of the data audit system are pre- edit, edit, and validation procedures based on computer analysis to flag data problems or to identify data anomalies prior to entering data into the NEDS data base. The first step in the audit process is the pre-edit procedure, which is performed by the regional offices. The pre-edit program sorts the data sub- mitted by the state agencies and examines each record for gross errors such as missing year of record or illegal transaction codes. A summary of incoming data, including the number of each transaction type in the data batch, is produced for data control purposes. After the pre-edit data problems have been corrected, the regional offices run the data against an edit program. The edit identifies and rejects data records that do not meet minimum criteria for updating the NEDS files. Necessary corrections are either made by the regional office or referred to the originating state agency. Records that pass the edit are forwarded to NADB 43 ------- STATE AGENCY RtOIONAUOFFICE USERS START, Figure B-l. Data flow through AEROS data auditing system* Volume II, AEROS User's Manual Section 7.0.1, AEROS Data Auditing System, U.S. Environmental Protection Agency, Research Triangle Park, North Carolina, EPA-450/2-76-029, December, 1976. 44 ------- for further processing. The edit program produces a summary report of edit diagnostics showing the specific problems with each data record requiring corrective action. The edit diagnos- tics are oriented primarily toward insuring that key data ele- ments for further record processing (i.e. estimating and reporting emissions) are present and that key processing codes are valid (e.g. emission estimation method codes, SCC codes, etc.). Exam- ples of NEDS edit diagnostic checks include: 0 checking for valid control equipment codes 0 checking for valid SCC codes 0 checking for valid card action codes (e.g. a "delete" transaction code is valid only for cards 1,2, and 6 for point sources) 0 checking for maximum parameter values (e.g. bituminous coal maximum ash content is 25 percent) The edit program includes 35 diagnostics. NADB runs the edited data received from the regional offices against a post-edit program similar to the pre-edit run at the regional offices. Problems flagged during the post-edit run are referred to the regional office. Data that pass the post-edit are run against NADB's update program to update the master file. Validation reports are gen- erated during the file update procedure. The validation reports identify suspect data, and it becomes the responsibility of the regional offices to correct or verify those data. Validation checks performed during the update procedure subject the data to more indepth analysis than the edit procedure at the regional office level. The update checks are oriented toward internal data relationship validity. Examples of the types of validity checks that occur during the NADB update are: 0 reasonable range checks for key parameter values, e.g. checking to see that the control device efficiency is reasonable for the indicated control device — 0 checks for completeness of data, e.g. ascertaining that specific SCC codes are included for specific emission sources within a facility. 45 ------- 0 .check for reasonableness of .parameters combinations, e.g. .determining that 'SIC and SCC codes are compatible •Questions from $a.ta -users regarding pos-sibl'e data .anomalies are directed to NADB. NADB in turn prepares a memorandum for action by the regional office, which may or may not require input from the state agency from which -the questionable .data were .originally reported.. Questions regarding data completeness are handled simila-rl.y to questions regarding -validity. 46 ------- TECHNICAL REPORT DATA (Please read /nsiructions on the reverse before completing) 1. REPORT NO. 2. 3. RECIPIENT'S ACCESSION NO. 4. TITLE AND SUBTITLE Development of an Emission Inventory Quality Assurance Program 5. REPORT DATE June 1979 6. PERFORMING ORGANIZATION CODE 7. AUTHOR(S) David W. Armentrout 8. PERFORMING ORGANIZATION REPORT NO. 9. PERFORMING ORGANIZATION NAME AND ADDRESS PEDCo Environmental, Inc. 11499 Chester Road Cincinnati, Ohio 45246 10. PROGRAM ELEMENT NO. 11. CONTRACT/GRANT NO. 68-02-2585 T.O. 8 12. SPONSORING AGENCY NAME AND ADDRESS U.S. Environmental Protection Agency Office of Air Quality Planning & Standards Research Triangle Park, North Carolina 27711 13. TYPE OF REPORT AND PERIOD COVERED FINAL 14. SPONSORING AGENCY CODE 15. SUPPLEMENTARY NOTES EPA Project Officer: James H. Southerland 16. ABSTRACT This report discusses quality assurance in general and quality assurance related to emission inventories specifically. The inventory process is discussed in terms of its procedural elements Typical error types and remedies are discussed. A suggested approach for developing a quality assurance program for emission inventories is giver.. KEY WORDS AND DOCUMENT ANALYSIS DESCRIPTORS b.IDENTIFIERS/OPEN ENDED TERMS C. COSATI Field'Group Quality Assurance Emission Inventories 18. DISTRIBUTION STATEMENT 19. SECURITY CLASS (This Report! Unclassified 21. NO. OF PAGES 46 20. SECURITY CLASS (This page I Unclassified 22. PRICE EPA Form 2220-1 (9-73) ------- |