United States Environmental Protection Agency Office of Air Quality Planning and Standards Air Quality Assessment Division Ambient Air Monitoring Group 109 T.W. Alexander Drive Research Triangle Park, North Carolina 27711 Inter-Mountain Labs Gravimetric Laboratory Technical Systems Audit Sheridan, Wyoming November 19 - 20,2014 Page 1 of 19 ------- EXECUTIVE SUMMARY The Inter-Mountain Labs (IML) gravimetric laboratory in Sheridan, Wyoming, operates a gravimetric laboratory for measuring PM2.5 on filter-based media that meets and exceeds the requirements of 40 CFR Part 50, Appendix L. The weighing lab staff demonstrated technical expertise in laboratory analysis, recordkeeping, organization, and quality assurance. The staff shows competency in filter weighing as a whole. The weighing procedure used by IML follows 40 CFR Part 50, Appendix L, and the gravimetric laboratory is well designed to maintain stable environmental conditions as dictated by 40 CFR Part 50, Appendix L. From the audit team's observations, the laboratory measurement process appears to produce measurements that meet both the method and quality assurance requirements. However, there are a few areas of recommendation identified that should be considered and implemented. These findings highlight issues related to documentation, quality assurance, and training. INTRODUCTION On November 19 and 20, 2014, Greg Noah, Environmental Protection Agency (EPA), Office of Air Quality Planning and Standards, Stephanie McCarthy, EPA Region 4, and Joshua Rickard, EPA Region 8, conducted an audit of the gravimetric laboratory located at the IML facility at 555 Absaraka Street in Sheridan, Wyoming. The audit included data collected from 2012 through 2014. IML is a contract laboratory which provides numerous analytical services for several types of sampling media. One of the services IML provides is PM2.5 gravimetric analysis support for multiple state and local air monitoring organizations throughout the country. Technical Systems Audits (TSAs) of contract laboratories are typically not part of EPA's oversight role unless they are a part of a program funded by EPA. However, recent results of TSAs of other gravimetric laboratories conducted nationally have revealed quality issues which have resulted in the invalidation of significant portions of data for various agencies. Since IML provides analytical support for the PM2.5 programs of many state and local monitoring agencies, EPA decided it was prudent to conduct a TSA at the IML laboratory to ensure PM2.5 data quality. During the audit, weighing procedures, shipping and receiving activities, and data entry were observed; laboratory climate control data and database systems were investigated; and the quality assurance process was reviewed. The IML gravimetric laboratory's procedures and operations were reviewed against the PM2.5 federal reference method codified in 40 CFR Part 50, Appendix L, and the Quality Assurance Guidance Document 2.12 Monitoring PM2.5 in Ambient Air Using Designated Reference or Class I Equivalent Methods. The TSA was conducted using the guidance provided by the Quality Assurance Handbook for Air Pollution Measurement Systems, Volume II and the Gravimetric Weigh Lab Systems Review questionnaire. Page 2 of 19 ------- IML staff interviewed during the audit included: Tim Mendenhall, Air Science Manager Mary Hininger, Gravimetric Lab Supervisor Carol Bredin, Lab Technician Lynn Kirkpatrick, Lab Technician Joyce Burnett, Lab Technician Michael Butler, Data Systems Manager Michelle LaGory, Quality Manager Margaret Elliott, Quality Auditor Gravimetric Laboratory Facility The IML gravimetric laboratory is a large area consisting of the weighing room, an anteroom, and a shipping and receiving area. IML estimates that 18,000 filters are weighed at this gravimetric laboratory annually. The weighing room is a custom-built temperature and humidity-controlled room. It contains the weighing station, filter equilibration racks, and filter inspection area. The weighing room's temperature and humidity are monitored, logged, and controlled automatically. Weighing data and room condition data are saved to a laboratory network server which is backed up weekly. Entry to the weighing room is limited, but it is not locked. It is managed as a "semi-clean room" environment. The anteroom serves as both a storage area for cassettes and other supplies needed during the weighing and preparation of the filter cassettes, as well as a work area for the analysts. This area also acts as a buffer from ambient conditions to the environmentally-controlled weighing room. Outside of the anteroom is a large room containing shipping supplies and a table used for shipping and receiving. The area appears functional for unpacking and organizing shipments. It also contains a small refrigerator used to store samples that are awaiting processing into the weighing room. This refrigerator has a temperature logger that tracks the refrigerator temperature over time to ensure samples are stored at 0° to 4° Celsius (C), as specified by 40 CFR Part 50, Appendix L, Section 8. IML has another weighing room that has been used for other projects requiring PMio filter weighing. At the time of the audit, IML was re-outfitting and testing this laboratory against PM2.5 filter weighing requirements. This new weighing laboratory will substantially increase IML's capacity for PM2.5 gravimetric weighing. Filters are archived in two coolers located on site. These coolers are environmentally controlled to 0° to 4° C. The cooler conditions are tracked and continuously monitored. Weighed filters are archived in these coolers for at least one year. Filter Weighing Activities All filters undergo inspection, equilibration, weighing (pre- and post-), and cassette assembly in the weighing chamber. Records reflected that the room is cleaned on a Page 3 of 19 ------- routine basis, and the weighing station surfaces are cleaned before each weighing session. The microbalance used in the laboratory weighs filters to the microgram (O.OOOOOOg) and measurements are electronically sent to the weighing software. Extensive static control measures have been taken to ground the microbalance and weighing support equipment. No effects of static were noted during the filter weighing process during the audit. IML uses lOOmg, 200mg, and 450mg working mass standards to check the calibration of the microbalance during weighing sessions. These weights can properly bracket the expected weight range of common brands of 46.2 mm Teflon® filters. Balance checks are completed at the beginning and end of each weighing session and after every 10 filter weighings. Calibration certificates that were examined showed all microbalance, mass standards, and logging equipment used in the weighing room were calibrated in accordance with Quality Assurance Guidance Document 2.12. Filters are allowed to condition for at least 24 hours prior to pre- or post-weighing, and this time is tracked in the IML Database Management System (DMS). Weighing guidance described in Quality Assurance Guidance Document 2.12 is followed when practical. The IML DMS for the gravimetric laboratory is a custom-built software package that serves as the interface for the analyst and the repository for filter weighing data. The DMS incorporates many well-designed automated checks to ensure quality control (QC) is maintained in the weighing sessions. The system is designed in a fashion so that if a check or QC requirement is not met, the database automatically displays a notification and sends an email. Climate control data is recorded and stored on a separate system. All electronic data gathered for the weighing sessions are backed up weekly. Currently, data backups are stored on site; however, IML is in the process of creating a procedure and method to routinely electronically transfer this data to another IML facility in Sheridan. An inspection of temperature and humidity data for the past three years demonstrated excellent environmental control of the weighing room, in adherence to 40 CFR Part 50, Appendix L requirements. All weighing sessions reviewed showed that weighings for filters used for NAAQS determination occurred within the 20° C to 23° C and 30% to 40% relative humidity (RH) ranges as specified by 40 CFR Part 50, Appendix L, Section 8. IML uses a standard deviation (SD) calculation and a simple difference from the prior 24-hour mean to assess and document the laboratory control criteria of ± 2° C and ± 5% RH, as dictated by 40 CFR Part 50, Appendix L, Section 8. No environmental conditions during weighing sessions showed variability exceeding the ± 2° C and ±5% RH control requirements. EPA observed two laboratory analysts weighing filters. During the filter weighing process, the analysts: cleaned the filter weighing work station; accessed the filter weighing database and began a new weighing session; checked room temperature and humidity; weighed check standards; identified blanks; and weighed filters for shipment. The analysts were efficient and methodical working through the process. The analysts were knowledgeable about the weighing criteria that must be met during the weighing process. Page 4 of 19 ------- Shipping and Receiving All incoming and outgoing shipment activities occur in the room immediately outside of the anteroom room. Outgoing shipments of pre-weighed filters are logged, packaged, and verified. Incoming shipments are logged, checked for integrity, verified against the chain-of-custody, and receipt temperature is recorded. The shipment temperature check is conducted using an infrared (IR) thermometer which is calibrated annually using a NIST-traceable standard. A sample receiving checklist is completed for every shipment by the analyst. Generally, filters are brought into the weighing room immediately after unpacking for equilibration. If they need to be held overnight, the filters are stored in the refrigerator located in the shipping and receiving area. The analysts are efficient and careful in their handling of the shipments, and the procedure is acceptable for handling the filter cassettes. Quality Assurance IML has an independent Quality Assurance group that conducts internal audits for the gravimetric laboratory. The group is knowledgeable of the PM2.5 reference method (40 CFR Part 50 Appendix L), Quality Assurance Guidance Document 2.12, and IML standard operating procedures (SOPs). The audits include a review of procedures, instantaneous checks of logging equipment, observations of activities, and a review of data. Copies of these internal audits were reviewed during the TSA. IML has an internally-approved Quality Assurance Project Plan (QAPP) for the gravimetric laboratory titled Quality Assurance Project Plan for Laboratory and Data Management Support of the Determination of Fine Particulate Matter as PM2.5 and Coarse Particulate Matter as PM10-2.5 in the Atmosphere, Revision 13, January 31, 2013, which also includes the gravimetric laboratory's SOPs and current laboratory equipment certifications. These documents are revised annually. These documents were reviewed by the EPA auditors and discussed. Quarterly control charts of QC data are also generated which graphically show: weighing room conditions, balance checks, replicate weighings, laboratory blanks, and refrigerator temperatures. Annotations are included on the charts which provide detail regarding the excursions shown on the charts. The charts are included with client data packages and these were reviewed during the audit. IML has a training program that includes a review of the QAPP, IML SOPs, CFR methods, and quality assurance guidance documents. The training program also includes hands-on training under an authorized trainer and practical demonstration of competence. IML Training and Authorization forms for the gravimetric lab were present for all gravimetric laboratory staff and were reviewed during the audit. Page 5 of 19 ------- AUDIT FINDINGS AND RECOMMENDATIONS EPA made several observations and recommendations during the audit. Please see the Gravimetric Weigh Lab Systems Review questionnaire included in Appendix A of this report for complete details of the audit findings. Significant commentary is summarized below with EPA's response reflected in bold type. 1. Neither the analysts nor the DMS calculates a difference between pre- and post- weighing sessions to ensure the filters meet the ± 5% difference stated in 40 CFR Part 50, Appendix L, Section 8.3.3. This is a critical criteria due to its inclusion in the CFR and its reference in the QA Handbook. However, an on-site assessment of the climate control percent relative humidity averages over the three-year period showed strict laboratory control at the room's set point from session to session with minimal variability. Also, standard deviations were well within the control criteria stated in 40 CFR Part 50, Appendix L, Section 8.2.4. Nevertheless, this is a critical element that must be tracked in the DMS and communicated to the weighing analysts through the DMS. EPA requires that IML develop a calculation in the DMS that documents this criteria and informs the analyst of the result. 2. IML uses two methods to show the laboratory is meeting the control requirements specified in 40 CFR Part 50, Appendix L, Section 8: a SD calculation, and a simple ± 2° C limit from the prior 24-hour temperature mean and a ± 5% RH limit from the prior 24-hour RH mean. While both are suitable for demonstrating control, one or the other should be chosen as the acceptance criteria to avoid confusion and to ensure the data is treated consistently. EPA recommends that IML determine which control method should be consistently used, revise the Quality Assurance Project Plan, and implement the change. 3. IML creates several quality control charts that help in assessing their laboratory's operation. Two of these charts are quarterly plots showing a trace of the weighing room temperature and relative humidity conditions and control statistics. The chart is created using hourly averages plotted across the quarter. Hourly averages mask variability and transient spikes in the dataset that can indicate potential issues and create a false sense of security. EPA recommends that IML plot five-minute averages to better display variability in the laboratory. 4. While reviewing balance check control charts, unusual shifts in the data were identified. IML uses a double substitution calculation quarterly to re-determine the mass of the working standards, which is not the correct use of this calculation. The double substitution method is a method that weighs a set of primary standards against a set of working standards to generate a reference point. Whenever the double substitution is computed, the new calculation is compared to the previous to determine if there has been a shift in mass. This method is not intended to calculate a "new mass" for a standard, which resultantly caused the shift in balance check data observed in the control charts. EPA recommends that IML Page 6 of 19 ------- cease using the double substitution method for creating a new mass, and instead use the calculation as a quarterly verification against the primary standards. 5. 40 CFR Part 50, Appendix L, Section 10.10 states that filters shall be removed from the samplers within 177 hours from the end of the sampling run. This time period is not being tracked in the IML DMS. While not a laboratory validation element, this is an important CFR field element for data validation, and it should be reviewed for any clients who contract for data management and analytical services. EPA recommends that IML track this criteria and use it in their data management services. This change would require an update in the QAPP. 6. The IML QAPP was revised by IML staff on January 31, 2013, and was reviewed by the EPA auditors prior to the audit. Some areas for improvement were identified and are listed below: ¦ Add pre- and post- ±5% RH comparison check ¦ Track 177-hour sample pick-up time criteria ¦ Review data verification tables, current ranges are too wide to be useful ¦ Provide detail on data review procedure or develop data review SOP ¦ Correct record retention time contradictions ¦ Update the NAAQS standard references to the new standard (12|jg/m3 annual mean) ¦ Update SOPs ¦ Add sliding scale weighing formula and how it is implemented at IML. 7. The analysts do not wear gloves or use a laboratory coat to protect against particulate contaminating the filters. The weighing room is maintained as a "semi-clean room" to minimize the chance of particulate contaminating the filters. The practice of wearing gloves and a coat is considered "best laboratory practice" in reducing the chance of contamination from the analysts themselves. EPA questioned the gravimetric lab supervisor regarding the lack of gloves and a lab coat. She responded that gloves reduced the dexterity of the analysts, increasing the chance of a dropped filter, and that quality control blanks were very low. EPA examined the quality control data and acknowledges that the levels are consistent or lower than other laboratories which require the use of gloves and coats. However, EPA maintains that IML should use anti-static gloves and lab coats when handling filters in keeping with "best laboratory practices". 8. The analysts use custom software to track weighing session data. The data recorded is comprehensive, and includes parameters such as filter weights, filter identification, dates, times, and filter flags. However, the temperature and humidity averages are not displayed on the user interface of the software. The software performs the weighing room prior 24-hour temperature and RH means and SDs in the background, and will alert the analyst if conditions are not Page? of 19 ------- acceptable to weigh. If conditions are not suitable for weighing, the software will lock out the analyst to prohibit work. The analyst checks present conditions through a different system, but the analyst cannot see the actual background calculations for the prior 24 hours. This observation has been noted in prior audits and should be a priority to ensure filters are weighed under required conditions. As a quality assurance control check, EPA recommends that IML modify the software to show temperature and humidity so the analyst can verify prior 24-hour conditions. 9. While reviewing training records and interviewing the laboratory analysts, the analysts demonstrated a thorough understanding of the procedures and methodology. However, knowledge regarding why the QC checks are important and what they represent was lacking. Training should include rationale on why a check is completed, what it represents, and the effects that failing a check has on the data. Understanding these elements ensures the quality of a program and diminishes the risk that a problem exists and is not identified. There also appeared to be a gap in understanding the "big picture" of why the PM2.5 data produced by the gravimetric laboratory is important and the ramifications of deviating from the method. EPA recommends that IML add "big picture" information and QC data to the training regimen to strengthen its program. 10. A refrigerator is present in the shipping and receiving area that is used to store incoming samples at 0° to 4° C as required by 40 CFR Part 50, Appendix L, Section 8.3.6. A logger is used to track the temperature of this refrigerator, but the logger is not verified for accuracy. EPA recommends that IML use a traceable thermometer to quarterly verify this logger. 11. At a client's request, IML may be contracted to conduct data validation and create files for upload to EPA's Air Quality System (AQS) database. IML's data validation is limited by the information the client provides. For example, along with gravimetric laboratory QC, field observations and checks such as sampler flow checks, calibrations, audits, maintenance records, and logbooks would be necessary to validate a dataset with confidence. Without this data and documentation, there are limitations as to what can be validated. Furthermore, 40 CFR Part 58.15 states that each agency is responsible for certifying its own data according to the QA findings. In short, air monitoring agencies hold the final responsibility for data review and validation before upload to AQS. EPA recommends that IML inform the clients requesting data validation of the limitations of this service and refer them to the language in 40 CFR Part 58.15. 12. IML uses an IR thermometer to measure incoming shipment temperature. The IR thermometer has been certified annually against a traceable standard; however, the certification sticker on the thermometer indicated that the unit had exceeded its annual certification. After reviewing certification records, a current certificate for the IR thermometer was found along with a current certification sticker. EPA Page 8 of 19 ------- recommends that IML remove the expired IR thermometer certification sticker and replace it with the current sticker. Page 9 of 19 ------- Appendix A Gravimetric Weigh Lab Systems Review questionnaire Page 10 of 19 ------- GRAVIMETRIC WEIGH LAB SYSTEMS REVIEW Analyst: IML Air Science Audit Date: November 19-20, 2014 Location: Sheridan, WY Auditor(s): Noah, McCarthy, Rickard Audit Questions Response Comments I. Routine Operations & Site Housekeeping List any visible sources that may impact the weigh lab. Is the room access restricted? Is the equipment located inside the weigh room only that which is required for daily weighing operations? Is there an anteroom? Describe its condition. What is being done to control drafts in the weigh room? Is the microbalance located so that it is not impacted by drafts? Is the weighing table stable so vibrations do not affect the balance? X Access to the PM25 weigh room is not restricted within the building; there are no locks on the doors, so any IM L employee can enter. Access is restricted after hours, when the main office building is locked. Inside the weigh room, a ceiling fan operates on low speed. It is a small interior room, located between the large filter receiving area and the PM 2 5 weigh room. IM L staff refer to it as the "handling room". The anteroom is clean and well organized. The room contains bins filled with petri dishes. Each bin represents a client. Upon receipt of exposed samples from the field, the filter containers are disassembled within the anteroon & exposed samples returned to their designated petri dishes in preparation for equilibration. IML staff have added a diffuser/deflector to the air conditioning unit within the weigh room, in order to minimize impacts on the balance. The weigh table is concrete, covered with an anti- static mat. Is the balance checked to determine if it is in fact leveled? If so, at what frequency? Visually inspect balance to determine if level. X The auditors observed that the balance was level during the audit. The analy sts look at the level occasionally. If the level is not balanced, the Air Science M anager (M endenhall) is contacted. Is the balance grounded? X Copper wire runs along the perimeter of the weigh room, with attachments connecting to all major items, including the balance and anti-static mats. IM L has done an excellent job grounding the weigh room. Is the balance left in the "On" position at all times? X Is the autocalibration feature on the microbalance on or off? What is the readability and repeatability of the balance? Is the balance certified annually by an outside source? When was the balance last certified? Where are the records of this maintained? Request copy of documentation. What anti-static prevention device(s) is in place? Are polonium strips used to eliminate static? Are they purchased in advance? If so, how far in advance are they purchased? How often are they replaced? What is stamped on the current strips? The analyst was uncertain about this feature, as were the Gravimetric Lab Supervisor (Hininger) and Air Science M anager. The user manual readily on hand was not for the balance model in use. Sartorius MC-5, S/N 90710302. Readability ±1 jag. Repeatability ±1 jag. The balance was serviced on September 16, 2014. Certification performed by Certified Balance Service (CBS), Littleton, Colorado. Records are maintained by the Gravimetric Lab Supervisor. Polonium strips are utilized within the weigh room, as are anti-static mats. Numerous grounding wires were visible within the room. The analysts also utilize Chemtronics Static-Free Spray when cleaning the work space prior to weighing Polonium strips are typically ordered 6-8 weeks prior to expiration. The strips expire within 6 months, so orders are typically placed -twice per year. New strips were received & placed in the weigh room the day prior to this audit. Page 11 of 19 ------- What device (LIMS, datalogger, etc) is used to monitor temperature and RH readings? What is the I resolution of data collected? What procedure is utilized to verify these readings and at what frequency ? Is the datalogger calibrated/certified annually ? When was the last certification? Where are the records maintained? Request copy. At what frequency are the temp erature and humidity sensors certified by an outside source? When was the most recent certification? Where are these records maintained? Request copies of documentation. Is the temperature maintained at 20-23°C, with a temperature control range of < 2°C over a 24-hour period? How is control demonstrated? I Campbell Scientific datalogging system, used in Iconjunction with a Vaisala temperature & relative I humidity probe. Second and hourly data are collected I & stored. However, in the weigh room, the analyst I sees 10-minute data on the electronic strip chart. I There is no specific calibration procedure conducted I by IML staff on the Campbell Scientific datalogging I system. The Gravimetric Lab Supervisor compares I the Vaisala temperature & relative humidity probe (responses (from the datalogger) to a NIST-traceable I hygrometer and thermometer on a monthly basis. I Results of these verifications are recorded in a I logbook, maintained within the weigh room. I The above-mentioned audit standards are certified on I an annual basis by Chinook Engineering, a division of I Inter-M ountain Laboratories (IM L). The most recent I certifications were on March 11, 2014, for the (hygrometer and August 5, 2014, for the mercury-in- I glass thermometer. The Gravimetric Lab Supervisor (maintains these records. Copies of certificates for the I past three years for each device were provided to the I auditors. Data is control-charted. Additionally, the IM L data management system (DMS) is programmed with the acceptance criteria; warning flags appear in the software system whenever parameters exceed specifications. IML staff also receive emails from the software system alerting them of any exceedances. The software displays a strip chart in the weigh room using 10-minute averages from the data collected. However, the weigh analyst is not fluent in the climate control specifications for the weigh lab. The analyst stated she will weigh unless instructed not to do so by management (or via an email alert). Is the relative humidity maintained at 30-40%, with a standard deviation of <5% over a 24-hour period? How is control demonstrated? How and where is temperature & RH data review documented? Frequency? Are pre & post sampling RH differences calculated? Where is this documented? See above. Data is control-charted. Hourly averages & minute data drive the decision-making processes in the weigh room on a daily basis. I In the DM S and through the control charts develop ed Iby the Data Systems Manager (Butler). If the temperature or RH is found to be out of sp ecification, what corrective action is taken? Are weigh sessions halted? Are maintenance/service contracts in place for climate control unit, sensors, and software in the lab? The pre- & post-RH differences are not currently calculated at IM L. However, a review of data showed that RH is tightly controlled and no deviations between pre & post-weighs were observed. IM L staff acknowledged that this is an area where they intend to improve and enhance both their DM S and documentation. If temperature or RH are out of specification, a warning screen will appear on the weigh room computer. An email alert will also be sent to staff. Upon such notification, the analyst will wait until the room returns to proper specification before weighing (-24 hours). The climate control equipment and sensors are maintained and routinely verified by IML. Spare heaters and humidifiers are available on site, if needed. The DM S was develop ed in-house and continues to be maintained and up graded, as necessary. Page 1 2 of 19 ------- Is electronic data backed-up at a defined frequency ? How long? Where is it stored? Print and review temperature and RH graphs for three prior weigh sessions (one for each year under review). Obtain the 24-hr means and SDs for temp and RH for those sessions. Are there two sets of weights being used: a primary set and a working set? Are they at least Class I weights ? See IML SOP# ML-AppLO314-1.0 for a description of the data back-up process. Additionally, IM L IT Procedures 1.1 document (August 29, 2011) further describes back-up policies. The DMS is backed up nightly to a virtual server. Weekly back-ups are completed manually using an external hard drive, which is taken to an offsite location. Troemner Class U weights are utilized. The primary set is clearly labeled as such, and stored (covered) on top of a shelf within the weigh room. The working (precision) weight set is kept out, with individual weights placed on the Polonium strip bars next to the balance. Do the weights bracket the weight of the filters being utilized? How frequently are the weights recertified by an outside source? Where is this documented? Request cop ies of documentation. At what frequency are the working weights verified against the primary weights? Where is this documented? Request copies of documentation. Who is responsible for ensuring that all standards are certified at their required frequencies? How is this tracked? Is a logbook used to document environmental conditions, weighing sessions, balance checks, weight checks, lot blank stability test results, etc, as well as lab maintenance activities? If not, how and where is this information documented? Are entries signed and dated? On what frequency and how is the weigh lab cleaned? Describe daily, monthly,&yearly cleaning regimes. How are sample cassettes, including stainless steel I backing screens, cleaned? At what frequency are I they cleaned? Are they inspected for cracks or other! damage? Do both halves fit tightly together when I assembled? I Is there a separate location designated specifically for cooler packing^unp acking? If so, describe its location and condition. 200mg and 450mg weights are utilized. The primary weights are recertified by an outside source on an annual basis. The Gravimetric Laboratory Supervisor maintains the certificates of calibration in binders. The working weights are verified against the primary weights on a quarterly basis. The results of the verifications are documented on the IM L Working Standard Certification form. The Gravimetric Laboratory Supervisor tracks standards & ensures all equipment utilized is within certification. She maintains logbooks and p ap er files containing the records for all certifications. There are three logbooks in the weigh room: The PM 2.5 Lab M aintenance Log, the PM 2.5 Lab Conditions Log, and the Sartorius MC-5 Balance Log Entries are sigued and dated. The weigh room is cleaned on a daily, weekly, and quarterly basis. See IML SOP#ML-AppLO304-1.0 for daily and weekly regimes, and SOP# ML- AppL0305-1.0 for quarterly activities. Cassette cleaning occurs weekly. The cassettes are p laced in a mesh bag. The mesh bag is then p laced in a bucket, along with a few drop s of Liquinox detergent. As for inspecting the cassettes, IMLhas no specific regimen. Cassettes are checked periodically for cracks and that there is a firm seal between the cassette halves. All PM 2.5 activities occur downstairs, in the basement of the building Sp ecifically, cooler (s amp le) receiving/unpacking occurs in a large, open room adjacent to the anteroom. This workstation contains a large, centralized table, which is used to unpack the coolers. Multiple shelves line the room. There is a small refrigerator in the corner. Page 13 of 19 ------- II. Sample Conditioning Where are new lots of filters stored once they are received from EPA? Describe the lot blank test procedure. What were the results of the most recent lot stability test? Request copy of documentation. Filters are primarily received from clients. The Filter Lot Login Sheet is documented upon receipt. Auditors observed filters stored in the weigh room. See SOP #ML-AppLO301-1.0 for more information. Three filters are selected at random from three random boxes within the lot. The Filter Lot Stability Evaluation form is documented. See SOP# M L- AppL0302-1.0 for details on the procedure. IML analysts conduct this procedure when instructed by the Gravimetric Lab Supervisor. Stabilization time -24 hours. Test conducted in M arch 2014. Documentation provided to auditors. Are all filters visually inspected for defects both pre and post-sampling? What technique is used to inspect the filters? What criteria would cause a filter to be rejected? Give examples. X Pre-exposed filters are examined on a light table and under a magnifying glass. Defects such as pinholes and discoloration would cause a filter to be rejected. Exposed filters are examined; however, they are not viewed on the light table or under the magnifying glass. Are all filters pre-conditioned prior to both the initial and final weighing sessions? How long is the conditioning p eriod? Where are they conditioned? If on a metallic shelf, is it grounded? Filters are conditioned for a minimum of 24 hours; filters can be conditioned over the weekend. Filters are placed in small plastic storage drawers in the weigh room during equilibration. There is airflow over the filters in the storage drawers. How is the conditioning period affected if the weigh I room conditions are out of tolerance? I Weighing is postponed if the environmental conditions I in the weigh room are out of specification. Are the filters conditioned in petri dishes or slides? Are the lids on the slides or are they slightly ajar? iFilters are conditioned in open petri dishes. Lids are I placed underneath the dishes. III. Sample Weighing Is one person designated to weigh all sample filters? Is there a back-up analyst? Does the same analyst weigh the same filters pre- and post-? There are three analysts in this production laboratory (Kirkpatrick, Bredin, & Burnett). The same analyst does not weigh the same filters for both pre- & post- weigh sessions. Because the analysts do not track & weigh the same filters each time, an inter-comparison study is conducted routinely to determine how much variability exists between the three analysts. Results of the most recent inter-comparison study were shown to the auditors; variability between the analysts was at a minimum. Are filters sampled within 30 days of the initial (tare) weigh? How is this tracked? The sticker on each individual sample bag has a "use by" date. It is the responsibility of the client to ensure the filters are used by the date on the sample bag (i.e., within the 30-day time frame). Clients are encouraged to use filters on a first in, first out basis. Are samples pre-assigned to a site? Or, is a site assigned after a filter is deployed? Elaborate on how filters are requested and distributed. IML prep ares the filters provided by clients. The filters are assigned to the individual sites of those clients. However, the client's field technician is ultimately responsible for determining the day on which each filter runs at the particular site. Page 14 of 19 ------- How are filters prepared for field deployment (i.e., loaded into sample bags or a magazine)? Where does this activity occur? Are samples maintained in a secure area at all times after being delivered to the laboratory ? How are filters packed for shipment back to the laboratory? What happens if a shipment (package and/or individual filters) is damaged in transit? Where are samples unloaded from the transport containers? Is this area clean and secure? Is the temperature of the cooler recorded at receipt in the laboratory? Is the temperature device NIST- traceable? Where is the temperature reading recorded? Describe the post sample weigh time limits (i.e., days/30 days) established for the lab and sup p orting rationale. Is a T-TM S sy stems used to record the weighing results? If not, describe how each weighing session is documented. If samples are refrigerated, is the temperature of the I refrigerator monitored? How is this accomplished? Is the monitoring equipment verified/certified? Filters are loaded into individual sample bags. Coolers are delivered to the sample receiving room (which is the large, op en room adjacent to the anteroom). Access to this area is not restricted. I Post-exposure cooler packing is the responsibility of I the client. Clients utilize the carrier of their choice ((typically UPS or FedEx). Clients pack coolers with I blue ice (or similar ice substitute), so coolers are I received by IM L cold. Some coolers contain min-max I thermometers, others don't. I In the sample receiving room. The room is not secure, I but it is clean. I IM L staff utilize a NIST -traceable IR gun for I obtaining cooler temp erature up on receipt. For those I clients who opt to use min-max thermometers, themin I max would be read in order to obtain the requisite I temp erature. The cooler temp erature is recorded on Ithe "Condition Upon Receipt" form (which is I attached to the chain-of-custody form that I accompanies the shipment). It is also recorded on a I separate color-coded label utilized by the analysts, Iwhichhelps them organize and track shipments. The I color-coded label & procedure is not included in the llMLSOPs. I Coolers received 4 degrees C or colder are allowed up I to 30 days. IML utilizes the post-sample weigh time I formula from the January 2000 OAQPS PM 2.5 I Cassette Handling memo that allows one to recalculate Ithe post-sample weigh date dependent upon the Icooler temperature. This procedure (and formula), I however, is not documented in the IM L Q APP or IsOPs. I The temperature within the sample refrigerator in the I receiving room is monitored. However, the I thermometer in that refrigerator is not verified on a I routine basis. IM L utilizes an in-house developed and maintained Data M anagement Sy stem (DM S). Are anti-static, powder-free gloves and lab coats worn while handling sample filters? IML analysts do not wear gloves when weighing They are not required to wear lab coats, although some of the analysts do. Are Teflon forceps used to handle the sample filters? How are they cleaned and at what frequency ? Are there separate forceps for "clean" versus "dirty" filters? The forceps used to weigh sample filters are stainless steel. The forceps are used for both clean and dirty filters. They are cleaned appropriately prior to use. Are the same forceps used for handling the mass standards used in handling sample filters? A separate set of forceps are used to weigh the mass reference standards. Those forcep s are Teflon-tipp ed, stainless steel. How often are balance checks performed? What is the tolerance (jag) for balance checks? How and where is this documented? Balance checks are performed prior to each weigh session, as well as every 10th filter weighed. The tolerance is 3 ^g. The results of the checks are documented in the DMS. See SOP #ML-AppLO306 1.0. Page 9 ------- If balance checks do not agree within ±3 jag, what corrective action is taken? How often are lab blanks weighed? If the lab blanks are not within +15 ^g, what corrective action is Foremost, the mass reference standards are reweighed. If there is still disagreement, an investigation takes place. The mass standards may be recertified, if needed. See SOP #IML-AppL0306-1.0. Lab blanks are weighed with each batch. The analyst interviewed during the audit stated that blanks are Are lab blanks used more than once? Are field blanks weighed with each session? Are final weigh results within + 30 of the initial weigh? How is this tracked? What corrective action is taken when FBs are out of limits? Field blanks are the responsibility of clients. Filters are weighed by IM L with no distinction as to whether filters are field filters or field blanks. The client will make the determination as to which filters are sample filters or blanks. IM L will notify the client of all sample results, including any blanks out of tolerance. It is the client's responsibility to conduct any needed investigations to determine the cause of elevated field blanks. Are duplicate filters weighed with each session? What is the acceptance limit (i.e., + 15 ^g)? Are sampled filters or blanks used? The analyst will choose one filter from a weigh session at random to reweigh. The acceptance limit is 15jjg Sample filters, and not blanks, are used for the reweigh. The auditors encouraged the lab analyst to select an exposed filter for the reweigh, if available during the weigh session. What happens if a weighed filter appears to be an outlier? How is it handled? How are re-weighs documented? The DMS will alert the analyst if the gross mass is high. The high mass will prompt a reweigh. The DM S software has a button/key for reweighs, which will label the reweigh as a "duplicate". The data will then appear on the "Duplicate" page/report in the software system. Are trip blanks utilized? If so, at what frequency ? What is the acceptance limits for trip blanks (i.e., + 15 ^g)? X Trip blanks are the client's responsibility. Similarto field blanks, IM L will weigh all filters in the same manner — it is left to the discretion of the client to determine the designation of the filters (i.e., field filter, field blank, or trip blank). Following each manual weigh session, is the weighing (batch) audited? If so, who does the audit? What percentage of filters are reweighed? What limits are used to determine good agreement? Weigh sessions are audited, when specifically requested by the client. In that event, 10% of filters from the weigh session are selected for reweigh by an independent analyst. Tolerance is 15jjg. See SOP# ML-AppL0306-1.0. Who does the lab analy st notify when discrep ancies are found and/or corrective actions are needed? After the final weighing has been completed, how are samples stored, and for what period of time are they retained? The analy st will first notify the Gravimetric Lab Supervisor (Hininger). If needed, the Air Science M anager (M endenhall) will be notified as well, or the Data Systems Manager (Butler). Samples are stored in the large, walk-in cooler that is maintained in the maintenance building adjacent to, but outside of, the central IM L office. Filters are maintained for a minimum of one y ear. Page 16 of 19 ------- Are chain of custody (COC) forms submitted by the field technician for each sample? Are they signed by all parties within the chain? Where are they maintained? Who reviews the COC forms? How are field flags/notes linked to the filter data? How are these notes communicated to the data reviewer? Once filter weighing is complete, is a report listing the sample concentrations generated for QA review? Does it list any void/flagged sample(s) and the reason for invalidating the sample(s)? Who is responsible for generating this report? Who is responsible for reviewing it? The Gravimetric Lab Supervisor reviews them. The chain-of-custody and sample receipt forms are stapled together & filed in-house. The supervisor will scan them, if requested by the client. Hard-copies are maintained for 10 years. Comments written on the chain-of-custody /sample receipt forms, or on the sample bag labels, are entered manually into the DM S by the analy sts. From there, the comments are linked to the data and will be seen by the data reviewer during verification/validation procedures. Please note: Sample bag labels contain pertinent data from each sample run, but there is no space on the label for the initials or signature of the individual collecting the sample/data. The auditors suggested to IM L staff that the samp le bag label be modified to include a space for the technician's signature/initials, in order to further document the life cycle of the samples/data. After the weigh session is completed, the analyst prints the results of the session from the DM S for an initial review by an independent analyst. The independent analyst reviews the report and signs it. The DM S will integrate tables & data from the weigh session with tables/data from the field/sample files, in order to p roduce additional rep orts. The DM S is programmed with verification/validation criteria and will flag samples that do not meet specifications. (See IM L Gravimetric Lab Q APP for tables of acceptance criteria and flags.) Final data is then reviewed by the Gravimetric Lab Supervisor. Are concentrations verified to ensure data entry & computations are correct? How many samples are reviewed per batch? Describe the verification procedure that is utilized. Computations are randomly checked by the Gravimetric Lab Supervisor. Are control charts used? If so, detail the types of control charts developed & how they are maintained. At what frequency are the charts reviewed & by whom? Where are these charts located? During the data review process, does the reviewer differentiate between critical and non-critical criteria | when flagging data? Describe this process. During the data review process, are sampler maintenance results, precision checks, and audit results reviewed to determine if any samples should be invalidated or flagged based on the results of these activities? The Data Systems Manager prepares control charts. Charts are maintained that illustrate the results of working standard verifications, replicate weighs, lab blanks, relative humidity & temp erature conditions in the weigh room, and temperatures of the sample archive cooler. The charts are prepared to represent results for each calendar quarter. Review of these data validation elements is the responsibility of the client. Page 17 of 19 ------- Are the filter, summary data, and interval data downloaded from the instrument for each sample run? Where is this data stored? Is the data reviewed | as part of the QA audit process? How are corrective actions addressed? Are forms filled out for corrective actions ? Who reviews them in the data validation chain? If corrections are made to data as a result of corrective actions, how is this documented & verified? Who is responsible for follow-up? Are exceptional events or impacts from nearby sources documented? Where? Once the data has been audited, are null codes and any qualifiers applied to the sample reviewed? If so. who is responsible for applying the codes/flags? Is this prior to the data being up loaded to AQS? I Filter files are downloaded from the sampler and sent (to IML for processing Thedatais entered into the I DM S and will be flagged if it exceeds acceptance (criteria. SeelMLQAPP. However, other data files— (such as sampler interval data — which are pertinent to I the data review process, are not reviewed as part of I the routine QA process by IML. Final data validation I is the resp onsibility of the client. I When issues are observed during the data I verification/validation process, the Gravimetric Lab (Supervisor will email the client of the issue/concern. I There is no specific corrective action report that is I documented. All field-related corrective actions (and (subsequent follow-up) are the resp onsibility ofthe (client. Any lab-related issue will be documented in ( weigh room logbooks. (Documentation of exceptional events is the (responsibility of the client. Issues may be noted on ( sample bag labels or field forms. The DMS will apply flags to sample data when acceptance criteria are exceeded. See the IM L QAPP. If the client has contracted IM L to provide data management services in addition to gravimetric analysis, IML will produce an AQS-compatible (pipe delimited) text file with final data results. That file will contain AQS null value codes or qualifiers. However, it is the responsibility of the client to perform the final data validation procedures for all sample filters &, subsequently, modify the AQS file provided by IM L, if needed. If the client has contracted IM L to provide analytical support only, then the client is solely responsible for the preparation of the AQS data file and final application of qualifier codes. Are internal p erformance & sy stems audits of the weigh lab and supporting equipment (loggers, balances, etc) performed? Who conducts these audits? Describe the review process and how the results are documented. How are staff notified of the audit results? Where are the audit reports filed? The Gravimetric Lab Supervisor audits the temperature and humidity sensors within the weigh room monthly. Results are documented in the weigh room logbook. The Quality Manager (LaGory) and the Safety Officer Quality Manager (Elliott) conduct systems audits of the weigh room. The review includes a comparison of IML procedures against Guidance Document 2.12 and 40 CFR 50, Appendix L. A written audit rep ort is generated from their findings. Auditors were p rovided cop ies of the IM L internal audit reports from the last three years. V. Other Does the laboratory operate under an approved QM P, QAPP, and SOP? What are the approval dates for the current revisions? What is the size of the particulate matter network for which this weigh lab is in operation? Describe the number of samplers and their operational frequency. Approximate number of samples weighed p er month/y ear. IM L maintains a comp any QA M anual — last revised in April 2014. As IML is not an EPA-grant recipient, submission of its QA Manual, QAPP, and SOPs for EPA approval is not required. IM L is a production laboratory with numerous clients across the country, including State/Local air agencies and industry. IML staff estimate approximately 18,000 samples are weighed each year. Page 18 of 19 ------- Does this weigh lab gravimetrically analyze samples collected for PM10 and/or lead? If yes, then describe any notable differences in procedures for the PM 10 and/or lead samples. Describe in detail the training the lab analyst has received. Who trained the analyst? Which courses have been taken? Has the analyst been trained on both QC operation of the lab, as well as data verification/validation procedures? Describe in detail the training the back-up weigh lab analyst has received. How often do they weigh filters? X IM L maintains a second laboratory dedicated to PM j, filter weighing. Separate quality documents are maintained for the PM 10 analy s is, adhering to the requirements of 40 CFR 50, Appendices B & J. I All analy sts are required to read the IM L QAPP and IsOPs, as well as 40 CFR 50, Appendices L & O, and (Guidance Document 2.12. Analysts receive on the job I training, specific to their applicable duties, and are (required to complete a Demonstration of Competence l(DOC). Copies of DOCs were provided to the I auditors. See above. The three analysts weigh filters weekly. Additional Comments on Audit: Review of the working standard verification control charts showed a distinct shift in results. Upon examination of thelML procedure (SOP # ML-AppL0311-1.0) and discussion with the Gravimetric Lab Supervisor, it was confirmed that IML changes the weight of the standards in the DM S per the results of the quarterly calculations. The auditors recommended this procedure be modified; the weight of the standard should not change in the database as a result of the verification check. Next, the IR gun in use was within certification — documentation was provided to the auditors. However, the certification sticker on the IR gun itself was outdated. Auditors suggested to IM L staff that the sticker be up dated on the IR gun with each certification. Additionally, during the course of this audit, interviews with staff indicated that refresher training as to the regulatory/big pictures requirements of the PM 2.5 method would benefit the analysts & improve the overall quality system of the company. Finally, IML staff were encouraged to provide clients with additional language on data reports to better explain the supplied control charts, as well as provide a disclaimer that clearly states final validation of PM 2.5 data is the resp onsibility of the client. Auditor's Signature: END OF REPORT Page 19 of 19 ------- |