United States
Environmental Protection
Agency
                                              EPA 530-R-16-008
                                                 February2016
                         Implementation Study of the
                   Electronics Recycling Standards:
                                    R2 and e-Stewardsฎ
                                                  Prepared by:
                         United States Environmental Protection Agency,
                         Office of Resource Conservation and Recovery
                                                         and
              Abt Associates, Inc. under EPA contract number: EP-W-10-054

-------
[This page intentionally left blank.]
                11

-------
Disclaimer
This document was prepared by the Office of Resource Conservation and Recovery (ORCR), U.S.
Environmental Protection Agency (EPA) and Abt. Associates, Inc. under contract number EP-W-10-
054. Any opinions, findings, conclusions, or recommendations do not change or substitute for any
statutory or regulatory provisions. This document does not impose legally binding requirements, nor
does it confer legal rights, impose legal obligations, or implement any statutory or regulatory provisions.
Mention of trade names, businesses or organizations, or commercial products is not intended to
constitute endorsement or recommendation for use. This document is being made available to the public.
Any questions or comments concerning this document should be addressed to Nicole L. Villamizar, U.S.
Environmental Protection Agency, Office of Resource Conservation and Recovery, 1200 Pennsylvania
Ave. N.W., MC 5306P, Washington, DC 20460 (email:Villamizar.Nicole@epa.gov).
                                             in

-------
Acknowledgements
The U.S. Environmental Protection Agency (EPA) would like to acknowledge the following organizations and
individuals for their substantial contributions to this document:

This study would not have been possible without the voluntary participation of the Standards owners - the Basel
Action Network and Sustainable Electronics Recycling International; the ANSI/ASQ National Accreditation Board;
the Certifying Bodies - NSF International Strategic Registrations (NSF-ISR), Orion Registrar, Inc., Perry Johnson
Registrars, Inc., SAI Global, SGS, TUV SUD America - and the electronics recycling facilities who agreed to
participate in this study.

The support team for this project was comprised of: Shanika Amarakoon, of Abt Associates, Inc.; Anne Peters, of
Gracestone, Inc; Nicole Delich, Kelley Keogh, Bob McCarthy, and Emily Farrant, of Greeneye Partners, LLC; and
Libby Chaplin, of Arcadian Solutions, LLC.
                                                 IV

-------
Table of Contents

List of Acronyms	vi

Glossary	viii

Executive Summary	1

  1. Introduction	10
    1.1.    Background	10
    1.2.    Study Purpose and Design	13
    1.3.    Study Limitations	16

  2. Study Findings	17
    2.1.    System Requirements	17
    2.2.    System Planning	18
    2.3.    Training and Communication	19
    2.4.    Operational Controls	21
    2.5.    Health and Safety	22
    2.6.    Reuse	25
    2.7.    Data Security/Data Destruction	26
    2.8.    Management of Focus Materials, Hazardous Electronic Waste, and Problematic Components and
           Materials	27
    2.9.    Emergency Preparedness and Response	28
    2.10.  Materials Recovery and Disposition	29
    2.11.  Export Restrictions for Focus Materials and Hazardous Electronic Waste	30
    2.12.  Site Closure and Insurance	32
    2.13.  Downstream Accountability	33
    2.14.  Monitoring and Measurement	34
    2.15.  Tracking	35
    2.16.  Evaluation of Compliance	36
    2.17.  Nonconformity, Corrective, and Preventive Action	37
    2.18.  Control of Records	38
    2.19.  Internal Audit	38
    2.20.  Management Review	39
    2.21.  Additional Findings from Stakeholder Interviews	40

  3. Data Analysis	41
    3.1.    Key System Strengths	42
    3.2.    Key Opportunities for Improvement	43

  4. Suggested Strategies for Improvement & Top Recommendations	45

  5. Conclusions	47

  Appendix A. Compilation of Suggested Strategies for Improvement by Topic Area	49

  Appendix B: Audit Observation Checklist	55

-------
List of Acronyms
 ANAB       ANSI/ASQ National Accreditation Board
 ASTSWMO   Association of State and Territorial Solid Waste Management Officials
 BAN         Basel Action Network
 CAPA       Corrective and Preventive Action
 CB          Certifying Body
 CRT         Cathode Ray Tube
 DOT         Department of Transportation (state or federal)
 EH&S       Environmental Health and Safety
 EH&SMS     Environmental Health and Safety Management System
 EPA         U.S. Environmental Protection Agency
 EPEAT       Electronic Product Environmental Assessment Tool
 ETBC       Electronics Take Back Campaign
 EU's  RoHS,   European Union's Restriction on Hazardous Substances and the Registration, Evaluation,
 REACH      Authorization and Restriction of Chemicals Directives
 FAA         Federal Aviation Administration
 EM          Focus Material
 FTC         Federal Trade Commission
 GSA         General Services Administration
 HEW        Hazardous Electronic Waste
 ISRI         Institute of Scrap Recycling Industries
 NAID       National Association for Information Destruction
 NCER       National Center for Electronics Recycling
 NFPA       National Fire Prevention Association
 NIH         National Institutes of Health
 NIOSH       National Institute for Occupational Safety and Health
 NSES        National Strategy for Electronics Stewardship
 OECD       Organization for Economic Cooperation and Development
 OEM        Original Equipment Manufacturer
 OFI          Opportunities for Improvement
 OSHA       U.S. Occupational Safety and Health Administration
 PACE       Partnership for Action on Computing Equipment
 PCM         Problematic Components and Materials
 PDCA       Plan-Do-Check-Act
                                                VI

-------
PPE         Personal Protective Equipment
SERI        Sustainable Electronics Recycling International
SDS         Safety Data Sheets
                                                vn

-------
Glossary
•   Acknowledgement of Consent: A notice issued by the competent authority of an importing country to the
    competent authority of country proposing to export used or scrap electronics containing hazardous waste, which
    documents acceptance of the hazardous waste by the importing country.

•   ANAB:  Short for "ANSI-ASQ National Accreditation Board"  - an impartial organization  that accredits
    conformity assessment bodies, such as testing and calibration laboratories, inspection, and certification bodies,
    to recognized standards.  ANAB ensures that Certifying Bodies are operating in compliance with practices set
    forth for Certifying Bodies in international standards, such as ISO  17021. In the electronics recycling industry,
    as of November 2015, ANAB has accredited 6 organizations to certify recycling facilities to the R2 and/or e-
    Stewards standards.

•   Audit Observation Checklist: Tool created by EPA to use in observing the audits of electronics recycling
    facilities seeking to obtain or maintain  certification to the R2  or e-Stewards standards. The Checklist is
    organized around twenty key topic areas that are addressed in both the e-Stewards and R2 standards,  such as
    Health and Safety, Reuse, and Site Closure and Insurance, and were derived based on all auditable elements in
    both standards. See Appendix B of this report for more information.

•   Audit Observer: The individual who observed a Certifying Body  audit an electronics recycling facility as part
    of this Study. Audit Observers received training to help ensure consistent performance across all  audit
    observations included as part  of this Study.

•   Basel Action Network (BAN): BAN is a non-profit organization that is the housing body, or owner of the e-
    Stewards standard.

•   Basel Convention for the Transboundary Movement of Hazardous Waste: An international treaty designed
    to limit the shipment of hazardous waste from developed countries to developing countries by requiring notice
    and  consent  from the countries engaged in the trade of hazardous waste. At the time of publication of this
    document, the United States was a signatory, but not a Party, to the Basel Convention.

•   Basel  Convention Ban Amendment: An amendment to the Basel Convention that prohibits the export of
    hazardous waste from a developed country to a developing country. As of the publishing date of this report, the
    Ban Amendment has not entered into force.

•   Bill  of Materials: List of the raw materials, sub-assemblies, intermediate assemblies, sub-components, parts
    and components needed to manufacture an electronic product, typically prepared in the manufacturing process.
    These are of interest to electronics recyclers wishing to identify (a) hazardous materials to design dismantling
    processes to  protect worker health and safety and the environment, (b) valuable materials to optimize value
    recovery, and/or (c) components or sub-assemblies having resale value to optimize reuse. Bills of Materials are
    often considered confidential  business information.

•   Certified Recycler: Recycler that has been certified to a recognized standard. In the context  of this Study,
    certified recyclers are those who have been certified to either the R2 or e-Stewards standards.

•   Certifying Body (CB): An organization that certifies electronics recycling facilities as meeting (or not meeting)
    the criteria in each of the electronics recycling standards (R2 and e-Stewards).  CBs are accredited to perform
    this activity by ANAB.

•   Competent Authority: The governmental authority of a country designated by that country to be responsible
    for receiving notification of transboundary movements of hazardous waste under the Basel Convention.  See list
    at www.basel.int/Countries/CountryContacts/tabid/1342/Default.aspx.

•   Conformance: Meets the requirement or provision of the Standard being audited.  Also see  'Generally
    Conforming Practice/Non-conforming Practice'  definition below.
                                                  Vlll

-------
•   Controls: When used in the context of certification, "control" refers to a procedure that is written to define how
    a task or process is to be completed. Typically it will include mechanisms for checking to make sure a task is
    completed.  There are several types of controls that can be used to achieve a desired environmental, health, or
    safety risk mitigation (e.g., engineering control - change a piece of equipment's configuration; administrative
    control - use administrative tools to address risk; or operational control - change the way technicians operate
    to address a risk).

•   Corrective Action: When used in the context of certification, an action taken to eliminate the cause of a non-
    conformity that has been identified in the facility's practices; corrective actions should be tracked by certified
    facilities.

•   Downstream Due Diligence: Investigation by an  electronics recycler of a downstream vendor to gather and
    verify needed information on the vendor's specific policies and practices, in conformance with the R2 and/or
    e-Stewards standards' requirements for downstream vendors and specific materials.

•   Financial Assurance: A mechanism put in place by an electronics recycler to ensure its facility can and will
    be responsibly closed in the event of default, business failure, unplanned closure, and/or abandonment. These
    mechanisms may include surety bonds, insurance policies, verified documentation of value of assets, or other
    financial instruments.

•   Focus Materials (FMs): As defined in the R2: 2013 standard, "Focus Materials contain: (1) Polychlorinated
    biphenyls (PCBs), or (2) Mercury, or (3) Cathode Ray Tube (CRT) glass, except for glass with lead content
    less than 5 parts per million, and clean of phosphors, CRT fines, coatings, and frit, or (4) Batteries, or (5) Whole
    or shredded circuit boards, except for whole and shredded circuit boards that do not contain lead solder, and
    have undergone safe and effective mechanical processing,  or manual dismantling, to remove mercury and
    batteries."  See the R2 Standard for full definition. Note that Focus Materials are not necessarily the same as
    Resource Conservation and Recovery Act (RCRA) designated Hazardous Waste. See the below definition of
    RCRA.

•   Generally Conforming Practice/Non-Conforming Practice: Auditor term for an area of activity in the
    standards that meets the requirement area ("generally conforming") or does not meet the requirement area
    ("non-conforming") of the standard being audited.

•   Hazardous Electronic Wastes (HEWs): As defined in the e-Stewards Standard v2.0, "Includes new or used:
    (a) Hazardous  Electronic  Equipment that is destined, or is intended to  be destined for: Recycling, energy
    recovery, or Final Disposal,  all or in  parts,  including shredded material, components, residues, and parts
    removed during Repair/Refurbishment, and/or Repair/Refurbishment or reuse, but not Direct Reuse, and (b)
    Electronic Equipment (including components)  that is: Tested and Fully Functional but for which a Direct Reuse
    market has not been affirmed according to requirements in 4.4.6.2 (Reuse), and/or Deemed hazardous waste or
    banned for importation by the country  of import or transit,  regardless of type of destination or  condition of
    equipment."  Note that the use of the term "HEW"  in this report explicitly refers to HEWs as defined by the e-
    Stewards Standard, and should not be  considered the same as a Resource Conservation and Recovery Act
    (RCRA) designated hazardous waste. See the below definition of RCRA.

•   National Strategy for Electronics Stewardship: A July 2011 report that lays the groundwork for how the
    federal  government can use its authorities and leverage resources to improve the design of electronic products
    and enhance the management of used or discarded electronics. See: www.epa.gov/smm-electronics/national-
    strategy-electronics-stewardship

•   Non-conformity (NC): Non-fulfillment of a requirement in the facility's management system. See 'Generally
    Conforming Practice/Non-conforming Practice' definition above.

•   Original Equipment  Manufacturer (OEM): In the context of this  study, a company that manufactures
    electronics (equipment or components), such as televisions, cell phones, tablets, laptops, computers, etc.
                                                  IX

-------
•   Plan-Do-Check-Act: Cycle of continuous improvement used in all quality and environmental management
    systems.1

•   Problematic Components or Materials (PCMs): As defined in the e-Stewards Standard v2.0,  "e-Wastes
    which may not be defined as Basel Convention hazardous wastes or e-Stewards Hazardous Electronic Waste,
    but which may be hazardous or require special controls..." See the e-Stewards standard for full definition.

•   R2/RIOS™: Combined certification to the R2:2013 Standard and to the Institute of Scrap Recycling Industry
    Operating Standardฎ (RIOS). RIOS defines an integrated quality, environment, health and safety management
    systems standard for the industry. R2/RIOS™ was created in 2013 to offer a certification path for electronics
    recyclers that combined RIOS  with the R2  standard.  When the R2:2013 standard emerged,  Provision l(b)
    mandated, "An R2:2013 electronics recycler shall be certified, throughout the duration of its R2 certification,
    to one or more environmental, health and safety management  system standards (EHSMS) that have  been
    approved by SERF' - it clarifies that SERI "has approved RIOS™, or a combination of both ISO 14001 and
    OHSAS 18001, to fulfill this requirement."

•   RCRA (Resource Conservation and Recovery Act) Hazardous Waste: The Resource Conservation and
    Recovery Act (RCRA) and its implementing regulations create  the framework for managing  hazardous and
    non-hazardous solid waste in the United States. In the U.S., a solid waste is considered a hazardous waste if it
    meets the listing criteria and  is listed in 40 CFR Part 261,  Subpart D, or if the solid waste exhibits the
    characteristic of ignitability, corrosivity, reactivity, or toxicity.

•   Root cause analysis: An analytical approach used to identify the underlying base of faults or problems. A
    factor is considered a root cause if its removal from the problem-fault sequence prevents the final undesirable
    event from recurring.

•   Sustainable Electronics Recycling International  (SERI): SERI is a  non-profit  organization that is the
    housing-body for the R2 Standard.

•   Universal Waste: The U.S. EPA's universal waste  regulations streamline hazardous waste management
    standards for federally designated "universal wastes," which include: batteries, pesticides, mercury-containing
    equipment, and bulbs (lamps).  The federal universal waste regulations are set forth in 40 CFR part 273. States
    can modify the universal waste rule and add additional universal waste(s) in individual state regulations, so it
    is important to consult with your state for the exact regulations that apply.

•   Witness Audit: ANSI-ASQ National Accreditation Board's (ANAB) observation of a Certifying Body's audit
    of an electronics recycling facility, in order to ensure the Certifying Body is following the requirements of the
    standard being audited.
1 For more about P-D-C-A, see https://en.wikipedia.org/wiki/PDC A or http://asq.org/learn-about-qualitv/project-planning-
tools/overview/pdca-cvcle.hrml.

                                                   x

-------
1.  Executive Summar
In the United States, electronics recycling facilities can be certified to standards through two certification programs
- the Responsible  Recycling  ("R2") Standard for Electronics Recyclers and the e-Stewardsฎ Standard for
Responsible Recycling and Reuse of Electronic Equipmentฎ ("e-Stewardsฎ"). Both certification programs were
developed to set minimum standards for ensuring worker health and safety, proper management of used electronics
though the recycling chain to their final destination, and data security. As of December 2015, more than 550 U.S.
electronics recycling facilities are certified to one or both of these standards.2

Under the 2011 National Strategy for Electronics Stewardship (NSES), which tasks the federal government to lead
by example in encouraging the greener design and responsible management of electronics, the U.S. Environmental
Protection Agency (EPA) committed to study - in collaboration with the General Services Administration and the
ANSI-ASQ  National Accreditation Board (ANAB) - the implementation of these certification programs. The
objective of the Implementation Study ("the Study") was to assess whether the R2 and e-Stewards standards ("the
Standards")  are being implemented transparently, consistently, and are achieving the desired results.
Study Desiqn
To  achieve the  objective of  the  Study,  EPA  conducted  interviews with
stakeholders across the electronics recycling system, observed audits of recycling
facilities seeking to obtain or maintain their certification to one or both of the
Standards to assess the consistency and rigourousness  of  the  audits; and
documented the findings, conclusions and recommendations in the form of this
final report. Figure ES-1 provides an overview of the timeline of the Study.

  Figure ES-1:  Timeline of Implementation Study
     Study Design
1.   Stakeholder Interviews
2.   Audit Observations
3.   Data Analysis
4.   Final Report
Study
Initiation
(10/12)
Stakeholder
Interviews
(3/13)
Audit
Observations
(8/13 to 3/14)
Stakeholder
Interviews
(5/14)
Audit
Observations
(10/14 to 4/15)
Analysis of
Findings
(4/14 to
6/15)
Final Report
Development
(6/15 - 1/16)
Stakeholder Interviews

EPA conducted  a total of 39 interviews with stakeholders across the electronics recycling system. Interview
questions were uniquely tailored to each stakeholder group in order to reflect their different roles, responsibilities,
and perspectives with regard to the electronics recycling certification process. The objectives of the first set of
interviews, conducted prior to observing any audits, were to establish baseline information about how the different
stakeholders across the system were implementing the Standards, identify potential issues of concern, and provide
an opportunity for the interviewees to share their perspective on how implementation  could be improved. The
stakeholders interviewed were:

       Representatives from R2  (owned by Sustainable Electronics Recycling International (SERI)),  and e-
       Stewards (owned by the Basel Action Network (BAN));
    •   An ANSI-ASQ National Accreditation Board (ANAB) representative;
       A national-level certified recycling facility;
    •   The  six  Certifying Bodies authorized to certify  recycling facilities to the Standards: NSF-ISR; Orion
       Registrar, Inc.; Perry Johnson Registrars, Inc.; SAI Global; SGS; and TUV SUD America; and
       The  Electronics Manufacturers  Recycling Management Co. (MRM),  which represents a consortium of
       electronics manufacturers.
2 Total number of certified recyclers as indicated on both the R2 and e-Stewards web sites.

-------
Additional interviews were conducted concurrently with the audits that were observed in order to obtain further
insight from the auditors and the facilities regarding opportunities to improve implementation of the Standards.
These post-audit interviews were always conducted outside of the formal audit to ensure EPA did not influence or
interfere with the audit. The stakeholders interviewed were: the nine Certifying Body auditors; two ANAB auditors
who were acting in their oversight role witnessing Certifying Body auditors conduct two audits; and the owners or
operators of the nine facilities whose audits were observed as part of the Study.

Finally, a subset of stakeholders from the initial interviews were re-interviewed in 2014 to better understand their
perspective on how the implementation of the Standards had changed with the issuance of the new 2013 versions
of the Standards. The stakeholders re-interviewed were: ANAB, five Certifying Bodies (NSF-ISR, Orion Registrar,
Inc., Perry Johnson Registrars, Inc., SAI Global, and SGS), and the Standards owners (SERI and BAN).

The information shared with EPA as part of the interviews was directly incorporated into the findings of the Study
and is reflected in Section 2 of this report. EPA did not undertake any additional activities to verify the information
that was provided beyond that which was stated or provided by the interviewees.

Audit Observations
Between August 2013 and April 2015, EPA observed a total of
nine audits of recycling  facilities seeking to obtain or maintain
their certification to one or both of the Standards: four e-Stewards
audits, four R2 audits, and one R2 and e-Stewards combined audit.
Five audits were to the 2008 versions of the Standards,  and four
were to  the 2013 versions of the Standards. In addition, two of the
observed  audits  were ANAB  witness audits, where  an ANAB
auditor  assessed a  Certifying Body conduct an audit as part of its
oversight  in observing  a  Certifying Body  auditor observed
auditors EPA observed  audits  performed  by  five of the  six
Certifying Bodies, which collectively are responsible for auditing
over 98% of the facilities that are certified  to one  or both
Standards.

The recycling facilities that participated in the Study represented
a cross-section of the different facility types, sizes, and ranges of
services offered by facilities that can be certified to the Standards.
Through the audit observations, EPA  identified both positive
actions being carried out as well as opportunities for improvement
across 20  key topic areas addressed in both of the Standards (see
text box to the right).  Because the Study intentionally does not
compare one Standard to the other, the 20 topic areas were derived
based on all auditable elements present in both the e-Stewards and
R2  standards.

EPA developed an "Audit Observation Checklist" to support the
assessment of the rigorousness and consistency of each of the
audits. The Checklist was organized around the 20 topic areas and
encompassed  144 discrete elements of the  Standards.  The
checklist  was used to rank the auditor's effectiveness  from  1
(auditor was  not effective or did not address)  to 5 (auditor was
interview  questions to ask  of  each key participant in the Audit
    20 Key Topic Areas Analyzed in the
                   Study
   1.   System requirements
   2.   System planning
   3.   Training and communication
   4.   Operational controls
   5.   Health and safety
   6.   Reuse
   7.   Data security/data destruction
   8.   Management of Focus Materials (FMs)
       and Hazardous Electronic Waste (HEW)
       and Problematic Components and
       Materials (PCMs)
   9.   Emergency preparedness and response
   10.  Materials recovery and disposition
   11.  Export restrictions for Focus Materials
       (R2) and Hazardous Electronic Waste
       (e-Stewards)
   12.  Site closure and insurance
   13.  Downstream accountability
   14.  Monitoring and measurement
   15.  Tracking
   16.  Evaluation of compliance
   17.  Nonconformity, corrective and preventive
       action
   18.  Control of records
   19.  Internal audit
   20.  Management review
highly effective).The Checklist also included
 Observation (CB Auditor; Facility; and, as

-------
appropriate, ANAB  Witness Auditors),  at the end of the  audit, to gain their perspectives  about the overall
effectiveness of the Standards' implementation. The Checklist is included as Appendix B of this report.

Data Analysis
Information from the interviews and nine observed audits was aggregated and analyzed to identify patterns of
strengths, opportunities for improvement, and other trends. With regard to the audit data, average ratings across all
nine audits for each of the 144 discrete elements of the Checklist were calculated, and those averages were then
weighted according to the number of audits in which each element was assessed.3 The weighted averages for each
of the elements were then used to calculate an overall score for each of the 20 Topic Areas (e.g. Health and Safety,
Reuse, etc.) to provide a quantitative sense of how well that particular area of the Standards was audited during the
observed audits. It is important to note that the auditing process is highly qualitative, and the average overall score
for each topic area reflects EPA's assessment of how well that particular topic area of the Standards was audited
based only on the nine observed audits.  The score is an indicator - but not the only indicator - of the overall
effectiveness/strength of that topic area.  The qualitative information provided through the structured stakeholder
interviews  also was  carefully reviewed to  determine trends,  such as frequently mentioned  strengths  and
opportunities for improvement, in the implementation of the Standards. Thus, EPA considered both the combination
of the audit findings and the interview responses in identifying patterns of strengths, opportunities for improvement,
and other trends.

Finally, it should be  noted that the Study intentionally does  not draw conclusions about whether one Standard is
being implemented more or less rigorously, transparently, or consistently than the other. The purpose of the Study
was  to conduct a system-level review  to assess  whether  the Standards as a whole are being implemented
transparently, consistently, and are achieving the desired results. As such, the interviews and audits focused on how
well the Standards are being implemented in 20 key topic areas present in both of the Standards.
Study Limitations
Table ES-1 provides a summary of the limitations of the Study, which are important to provide context for the
findings and conclusions, as well as for future studies that may be undertaken.
Table ES-1:  Implementation Study Limitations
Limitation
Snapshot in
time in a rapidly
changing industry
Subset of entire
universe of
electronics recyclers
studied
Limited sample size
Discussion
• Interviews and audit observations were conducted from March 2013 through May
2015, and only represent findings during that period.
• As of December 2015, more than 550 U.S. electronics recycling facilities were
certified to one or both of the R2 and e-Stewards Standards.4 As such, the
Study's findings should not be construed to apply to the full universe of
electronics recyclers operating in the U.S.
• The Paperwork Reduction Act limited the number of interviews and audits that
could be observed by the EPA as a part of this Study.5 The Study's findings are
based on the stakeholder interviews EPA conducted and the audits EPA observed,
which reflect a small percentage of all certified electronics recycling facilities and
of all auditors working in the electronics recycling industry in the U.S.
3 Not every element was assessed in each of the nine audits due to the requirements of different audit types (e.g. Stage 2,
surveillance, and recertification) and of the different versions of the two Standards (2008 and 2013).
4 Total number of certified recyclers as indicated on both the R2 and e-Stewards web sites.
5 The Paperwork Reduction Act was enacted to minimize the paperwork burden for individuals; small businesses; educational
and nonprofit institutions; federal contractors; state, local, and tribal governments; and other persons resulting from the
collection of information by or for the federal government.

-------
Limitation
Role of the
Audit Observers was
observational only
Facilities agreed to
let EPA observe
audits
Not all CBs observed
in audit portion of
the Study
No Stage 1 audits
were observed
Discussion
• EPA did not ask direct questions while observing the Certifying Body auditors
conduct the facility audits. Findings based on the observed audits are therefore
limited to what could be seen rather than discussed with the Certifying Body
auditors.
• For the nine audits EPA observed, the Certifying Bodies contacted their clients
(facilities) in advance, and those facilities all agreed to allow EPA to observe their
audits. It can be surmised that the clients had confidence in a positive outcome of
the audit in order to agree to let EPA observe the audit.
• EPA observed audits performed by only five of the six Certifying Bodies during
the Study. As of the writing of this report, those five Certifying Bodies are
collectively responsible for auditing over 98% of the facilities that are certified to
one or both of the Standards.
• Audit Observations were for facilities obtaining Stage 2 certification, surveillance,
or re-certification audits; no Stage 1 (readiness review) audits were observed.
Stage 1 audits primarily consist of reviewing facility documents in preparation for
a Stage 2 audit, where certification may first be achieved.
Summary of Findings
The Study's findings are based on the stakeholder interviews EPA conducted and the audits EPA observed, which
reflect a small percentage of all certified electronics recycling facilities and of all auditors at work in the electronics
recycling industry in the U.S. The interviews EPA conducted included all of the key entities that are critical to the
implementation of the Standards. Though limited in number, the audit observations reflect a cross-section of the
types of facilities that can be certified to the Standards in terms of the different types of audits (e.g., Stage  2,
surveillance, recertification), facility sizes (ranging from 5-40 employees; single facility vs. multiple facilities
across the country), and  services offered  (reuse, manual disassembly,  shredding, data destruction, etc.). As
previously noted, the data from the interviews and audits were analyzed to identify key strengths, opportunities for
improvement, and other trends, which are summarized below.

Strengths
The Study's findings suggest that overall, the Standards have brought order, better management, and a growing
understanding of regulatory requirements, best practices, and environmental, health, and safety risks to electronics
recyclers and related stakeholders throughout the system. In addition, the interviews found that stakeholders are
operating with a willingness to learn complex new subject matter,  such as legal and environmental compliance,
downstream  due  diligence  practices,  export  requirements,  and occupational  health  and  safety  practices.
Furthermore, interviews and audits indicated that the roles and responsibilities among the key implementers in the
system - ANAB, the Standards owners, the six Certifying Bodies, auditors, and recycling facilities - appear to be
clear and effective, and opportunities for constructive feedback are integrated throughout the system. Certifying
Bodies have developed and implemented effective training programs for their auditors covering the critical elements
of the Standards. Auditors were found to be very hard working and diligent, using their often limited time on the
audits well.

Table ES-2 below provides highlights of areas  of strength in the Standards that were observed to  be audited
comprehensively and consistently across multiple audits. The average overall score provided for each topic area is
derived from assessing all 20 topic areas across the nine audits observed.  As previously mentioned, the auditing
process is highly qualitative,  and the average overall score for each topic  area reflects  EPA's assessment of how
well that particular topic area of the Standards was audited based only on the nine observed audits.  The score is an
indicator - but not the only indicator - of the  overall effectiveness/strength of that topic area. The qualitative
information provided through the structured stakeholder interviews also was carefully reviewed to determine trends,
such as frequently mentioned strengths and opportunities for improvement, in the implementation of the Standards.
Thus, EPA considered both the combination of the audit findings and the interview responses in identifying patterns
                                                    4

-------
of strengths, opportunities for improvement, and other trends. Section 2 of this report provides a detailed assessment
of findings observed within each topic area, and Section 3 provides a comprehensive discussion of the data analysis
and aggregation process for identifying overall strengths and opportunities for improvement.  Table ES-2 is not
intended to be an inclusive list of all areas that were found to be strengths in the Standards.

Table ES-2: Key Areas of Strength
Topic
Area*
1.
7.
14.
7.
Topic Area Name
System
requirements
Data security/data
destruction
Monitoring and
measurement
Nonconformity,
corrective and
preventive action
Discussion
• Facilities are devising Environmental, Health, and Safety
Management Systems (EHSMS) that cover the entire
scope of the applicable Standard(s), and are ensuring that
EHSMS are kept up-to-date.
• The updated versions of the Standards have improved the
detail and requirements for this area, and very thorough
auditing of this topic area was observed.
• Auditors are ensuring facilities are monitoring and
measuring, on a regular basis, their activities that have
environmental and health and safety impacts.
• Auditors are ensuring the facilities assess internal
nonconformity with their EHSMS and are taking corrective
and preventive action.
Weighted
average
audit
score (1-5)
5.00
4.21
4.54
4.83
Interviews suggest that the continued evolution of the Standards and their implementation are helping to maintain
the integrity of the Standards and helping to ensure they are being implemented in a clear and consistent manner.
Both Standards were updated during the course of the Study, in the latter half of 2013, and the interviews conducted
and audits observed during and after the updates indicated the roll-out of the new versions went smoothly.  Key
improvements made to the Standards' text and implementation systems included:

    •   Updating training programs,
    •   Updating certain requirements (e.g., environmental, health and safety criteria), and
    •   Providing clarification or additional  guidance documents on existing requirements to aid in consistent
        implementation of the Standards.
Finally, with respect to the recycling facilities, the interviews and audit observations also found that they are well
engaged in the  certification process and view it as a benefit to their customers.  As a result, market demand for
certifications continues to grow among generators of electronic waste.

Opportunities for Improvement
The interviews and audit observations indicated that overall, the Standards are generally being implemented well;
however, opportunities for improvement in implementation were identified across all 20 topic areas assessed in the
Study. Key areas for improvement are summarized in Table ES-3 and were based primarily on the average overall
score for each of the 20 topic areas, with additional consideration of the qualitative information from the interviews.
As previously mentioned, it is important to note that the auditing process is highly qualitative, and the average
overall score for each topic area reflects EPA's assessment of how well that particular topic area of the Standards
was audited based only on the nine observed audits. The score is an indicator - but not the only indicator - of the
overall effectiveness/strength  of  that topic area. The  qualitative information provided through  the structured
stakeholder interviews also was carefully reviewed to determine trends, such as frequently mentioned strengths and
opportunities for improvement, in the implementation of the Standards. Thus, EPA considered both the combination
of the audit findings and the interview responses in identifying patterns of strengths, opportunities for improvement,
and other trends.

-------
Note that Section 2 of this report provides a detailed description of all of the observed opportunities for improvement
and Table ES-3 is not intended to be an all-inclusive list of opportunities for improvement.

 Table ES-3:  Key Opportunities for Improvement	
  Topic
  Area*
    Topic Area
                    Discussion
Weighted
 Average
  Audit
  Score
    3.
Training and
communication*
Auditor and facility training, understanding and consistent
implementation of:
 •   Environmental health and safety hazard risk reduction
     strategies;
 •   Hazardous waste requirements;
 •   Import/export requirements (especially for Cathode Ray
     Tubes (CRTs); Organization for Economic Cooperation
     and Development (OECD) countries vs. non-OECD
     countries);
     Federal, state, and local legal requirements (e.g.,
     Universal Waste rule); and
     Proper materials management (including batteries).
   4.32
          Operational control
                         Auditor verification of implementation of written procedures
                         such as how to manage FMs and how to best mitigate
                         environmental, health, and safety risks -during audits.
                                                           3.77
          Health and safety
                         Auditor and facility understanding and dissemination of
                         actionable information regarding human health risks posed by
                         dismantling end-of-life electronics, including potential
                         exposures and subsequent use of risk-based controls (e.g.,
                         personal protective equipment (PPE)).
                                                           3.87
    8.
Management of
FMs, HEWs, and
PCMs
Auditor and facility understanding of what is and is not
hazardous or universal waste under state and federal
regulations, and the associated management requirements.
   4.04
   10.
Materials recovery
and disposition
Facility and auditor attention to safe packaging of hazardous
equipment (e.g., CRTs, batteries).
Auditor verification of facilities' controls for ensuring
downstream processors conform to Standards' requirements.
   4.06
   11.
Export restrictions
for Focus Materials
(R2) and
Hazardous
Electronic Waste
(e-Stewards)	
                                   Auditor and facility training, understanding, and awareness of
                                   export requirements (particularly for CRTs).
                                                           3.00
   12.
Site closure and
insurance**
Auditor understanding of the form and accessibility of
insurance (or other forms of financial assurance) in the event
of abandonment or bankruptcy.
   4.44
   13.
Downstream
accountability
Auditor verification of requirements in areas such as:
confirming that all downstream material flows are identified
throughout the recycling chain beyond the first downstream
   3.85

-------
Topic
Area*

15.
16.
Topic Area

Tracking
Evaluation of
compliance
Discussion
vendor; and improving downstream verification practices by
auditors during an audit.
• Auditor performance of the mass balance calculation
(account for all transactions into facilities and all transactions
out of facilities) to verify equipment going for reuse.
• Auditor review of legal and environmental, health and safety
requirements, and compliance with those requirements.
Weighted
Average
Audit
Score

4.12
3.38
 * Auditors assess whether or not facilities have determined their training needs and communicate them
 appropriately. While these clauses were observed to be well audited, the need for further training and knowledge
 development was noted across numerous topic areas and emerged as a top area for improvement from the
 interviews.

 ** Auditors assess whether the closure plans are in place and contain the required components; however, while
 these requirements were generally audited well (meaning the auditors identified the closure plans with required
 elements), an important opportunity for improvement emerged to improve auditor understanding of the adequacy
 of the forms of insurance (or other forms of financial assurance).
Suggested Strategies for Improvement and Top Recommendations
The Study identified specific strategies for addressing the opportunities for improvement identified in each of the
20 topic areas assessed in the Study. These strategies are listed in tables at the end of each subsection of Section 2,
and are compiled in a summary table in Appendix A.

Top Recommendations
EPA also developed top recommendations to address the root causes of many of the opportunities for improvement,
which were developed based on the analysis of patterns and trends from the interviews and audits. In practice, many
stakeholders will continue to influence and shape the development of responsible recycling in the U.S. and therefore
have been considered when  drawing the  Study's conclusions  and making recommendations; as such,  EPA
encourages all organizations that have a role to play in shaping the standards and certification process to consider
the suggested strategies  for  improvement and top recommendations in  the  Study.  This broader group of
stakeholders includes:

    •   The Standards owners (Sustainable Electronics Recycling International and the Basel Action Network)
       The Certifying Bodies
       Federal and state regulators
       Trade associations
    •   ANAB
       EPA and other federal agencies, including the Occupational Safety and Health Administration,  the National
       Institute for Occupational Safety and Hazards, and the Department of Transportation
       Original equipment manufacturers
    •   Academic researchers
       Non-governmental organizations
       Private-sector companies

-------
It is important to note that because of the commitment to continual improvement by stakeholders in the system,
some of the below recommendations - as well as the suggested strategies for improvement presented in Section 2
of the report - may already be in motion.
The top recommendations are:
•  Provide additional training and guidance materials to grow the
   knowledge  base.  Growing  the  base  of  knowledge for  all
   stakeholders, including auditors and facilities, is important for
   ensuring the Standards are implemented properly. In total, 85%
                                                             '0
                                                                     Top Recommendations
                                                                   Provide additional training and
                                                                   guidance materials to grow the
of all interviewees - including 100% of Certifying Bodies and         stake'holders386
91% of auditors - indicated that more robust training programs     .   p^ ^Juiar updates to the
are needed in order to strengthen knowledge of health and safety
risks, legal requirements - particularly related to exports and
management of CRTs - and best operational practices in many of
the 20 topic  areas.  Combined with the  observations from the
audits and the low weighted average scores of many of the topic
areas, providing additional training and guidance materials to
                                                                      Standards to ensure they continue
                                                                      to evolve
                                                                      Increase audit time to allow for
                                                                      more thorough audits
                                                                      Explore and address perceived
                                                                      conflict-of-interest issues
    grow the knowledge base for all stakeholders in the system was
    determined to be one of the recommendations with the most far-reaching possible impacts in improving the
    overall implementation of the Standards. Section 2 of the Study offers suggested strategies for consideration in
    developing the additional training materials, guidance, and other tools in the context of each of the relevant
    topic areas.

•   Provide  regular updates to the Standards to ensure they continue to evolve alongside this rapidly changing
    industry.  E-Stewards and R2 are both relatively new standards and their recent updates went smoothly.
    Updating and revising the Standards on regular, well-publicized timetables is important to address areas that
    could benefit from clarification in a Standard or lessons learned from prior audits. Support for more systematic,
    well-publicized plans and/or timetables for subsequent updates was indicated in 69% of all interviews, including
    73%  of stakeholder interviews conducted before the 2013 Standards  revisions and 100% of stakeholder
    interviews conducted after the revisions. Scheduled and publicized plans to update the Standards  will allow
    stakeholders to fully contribute and participate in the continual improvement in a fair and transparent manner.

•   Increase audit time to allow for more thorough auditing of the Standards.   Inadequate  audit time was
    mentioned in 56% of  all interviews as  a limiting factor, particularly for integrated  audits where  R2 and e-
    Stewards are being audited at the same time as other standards, such as RIOS and ISO 14001. In interviews
    where the adequacy of the amount of time allowed for audits was questioned specifically, 75% of interviewees
    indicated a desire for increased audit times, as did five of the nine facilities audited. Audit time for ISO  14001-
    based standards is specified in "IAF MD 5: Duration of QMS and EMS Audits." To better reflect the complexity
    of electronics recycling operations, there is a need to fine-tune the formulas found in the R2 Code of Practices
    and e-Stewards Appendix C to 'right-size'  audit time.  Many facilities  can have multi-faceted downstream
    material flows, export  situations, regulatory status, reuse practices, and other variables that pose challenges to
    the most efficient auditors to adequately review materials and follow audit trails in the allotted time. Moreover,
    many audits cover multiple standards simultaneously. Increasing audit time would allow auditors to more
    closely examine health and safety monitoring, measurement, record-keeping, and communication issues in
    order to  give facilities critical feedback on their health and safety systems.

•   Explore and address perceived conflict of interest issues to enhance overall rigorousness of the audits.  Study
    participants expressed a concern that there is a perceived conflict of interest between CBs wishing to retain their
    clients (who are the recycling facilities) and the recycling facilities themselves; that is, the CB auditors may not
    be as stringent or thorough in  their audits in an effort to  retain a competitive business relationship. Similarly,
    auditors being observed  by ANAB Witness  Auditors may be more stringent in their findings, in an effort to
    prove competency and thoroughness; this inconsistent scrutiny may pose concerns to a facility getting more

                                                   8

-------
    non-conformances than it expected when  ANAB is present,  for example. When asked specifically about
    whether they feel that the perception of a conflict of interest exists, 63% of stakeholders interviewed said that
    they did. Exploring ways  in which the key players in the system - Certifying Bodies, recycling facilities, the
    Standards owners, and ANAB - could potentially address these perceptions of conflict of interest, such as
    implementing new or different funding mechanisms, could enhance the implementation of the Standards.
Conclusions
The  Study suggests that overall, the accreditation, certification and implementation process of the R2 and e-
Stewards standards is working well.  Through the interviews with stakeholders across  the electronics recycling
system and the observed audits representing a cross- section of facility sizes, types and services offered, the Study
suggests that the Standards have brought order, better management, and a growing understanding of environmental,
health and safety risks, regulatory requirements, and best practices to electronics recyclers and related stakeholders
across the system.

Though limited in scope, an  analysis of the  interview and  audit findings  for patterns  and trends identified
opportunities to improve  the implementation  of the  Standards  in a  few key  areas:  health and safety risks,
management, and  best practices;  awareness and understanding of all applicable federal, state and local legal
requirements; export requirements; and knowledge of hazardous substances in electronics. One central theme that
emerged from the  Study was that  the knowledge base needs  to continue to  grow and tools  are needed for all
stakeholders (e.g., for auditors to more efficiently audit, for facilities to more readily operationalize improvements
in these areas, and for regulators to better inform the regulated community). The Study offers four recommendations
that, if implemented, could address  many of the root causes of the opportunities for improvement, and Section 2 of
the Study also provides numerous  suggested actions that may be taken to address the specific opportunities for
improvement identified in all 20 topic areas assessed in the Standards.

EPA remains committed to continuing the dialogue started by this Study and supporting the continual improvement
of the implementation of the electronics recycling standards, and will provide assistance and support to stakeholders
in discussing and implementing the recommendations outlined in this report.

The following report presents additional  detail on the Study's background, approach, detailed findings (including
strengths and opportunities for improvement)  as well as final conclusions and recommendations.  A complete
summary of suggested actions to address the opportunities for improvement can be reviewed in Appendix A of the
Study.

-------
1.  Introduction
Electronics recycling began in the United States in the 1970s after the introduction of the first computers in the mid-
1960s. Electronic equipment manufacturers realized the potential profits from product resale and/or the recovery
of gold, silver, platinum, palladium and other precious metals, and thus began their own in-house recovery and
refurbishment/resale operations. It was not until the late 1980s/early 1990s, when computers and other electronics
were found on every desk in the workplace and in homes that non-manufacturer recycling companies started to
develop the electronics recycling industry as it is known today.

The rapid  growth of the industry  resulted  in  a need for  minimum standards  for worker  health and safety,
downstream due diligence  and data security.  Further, non-governmental organizations (NGOs) worked in the
early 2000s to raise awareness of the problems caused by exporting hazardous waste-containing electronic scrap
to developing countries. These NGOs were joined by regulators and responsible recyclers to jointly create standards
that would differentiate electronics recycling operations that were committed to (i) protecting worker health and the
environment,  (ii) preventing export of  e-scrap to destinations without adequate  worker and  environmental
protection,  and (iii) properly  sanitizing data.

Under the  2011  National  Strategy  for Electronics  Stewardship (NSES). EPA committed  to study - in
collaboration with the General Services Administration (GSA) and the ANSI-ASQ National Accreditation Board
(ANAB) - the implementation of the two electronics recycling certification programs currently used in the United
States: the Responsible Recycling (R2) practices and e-Stewardsฎ Standard for Responsible Recycling and Reuse
of Electronic Equipmentฎ (e-Stewardsฎ).6 The main goal of this Implementation Study ("the Study)  was to conduct
a system-level review of the accreditation, certification and  implementation process to determine whether the
Standards are being implemented transparently, consistently, and are achieving the desired results.
1.1.   Background
In order to evaluate the effectiveness of the implementation of the Standards, the Study examined all aspects of the
electronics  recycling certification process, including the Standards owners (Sustainable Electronics Recycling
International (SERI), which owns the R2 Standard, and the Basel Action Network (BAN), which owns the e-
Stewards Standard);  the ANSI-ASQ National Accreditation Board (ANAB), the six Certifying Bodies  (CBs)
accredited to certify facilities to R2, e-Stewards, or both; and a limited number of facilities seeking to  obtain or
maintain certification. The CBs, their auditors, and electronics recycling facilities were the main focus of the Study
as the entities most directly engaged with implementing the Standards; however, each entity in the system plays an
important role in ensuring effective implementation of the Standards. Each entity is described below, including its
role in the overall certification system, and how it was examined in the Study. Figure 1 illustrates the certification
process and the relationships among key entities.7
6 The NSES details the federal government's plan to enhance the management of electronics throughout the product life
cycle—from design to the eventual recycling or disposal of the products (see http://www.epa.gov/smm-electronics/national-
strategy-electronics-stewardship-nses).
7 For a more detailed explanation of how this system works, see "Understanding the Certification Process for End-of-Life
Electronics," December 2012, by Libby Chaplin, Arcadian Solutions, for EPEAT and the Green Electronics Council, at
http://arcadiansolutions.com/wp-content/uploads/R007-eole-certiFication-process-2012-12-21.pdf.
                                                   10

-------
                     Figure 1: Overview of the Certification Process and Relationships
                                           among Key Entities
              AKAB
        ANSI-ASQ National
        Accreditation Board

                                        Standard owners work
                                       with ANAB on oversight
                                           of the Standards
                        Standard Owners
                            R2 (SERI)
                        e-Stewards (BAN)
                   ;  ANAB creates
                   I  accreditation rules
                   •  for each of the
                   •  Standards and
                   |  ensures the CBs are
                   '  compliant
 Standard owners train
 CBs on the Standards,
   provide technical
 assistance, and make
X  interpretations
     I
  Facilities
implement &
 follow the
 Standards
     i
     I
     I

Accredited Certifying
Bodies (CB)

NSF
Orion
Perry Johnson
SAI Global
SGS
TUV SUD
e-Stewards
El
El
m
	 0 	


R-2
El
El
El
""g
El
El



t
CBs hire
and train
auditors
-$


•^
-*•
Auditors
r^*
&
i
•
L

$
? —
t —
i —
]
] —
^^H


Auditors
Conduct
audits to
determine
conformance
to the
Standard(s)

Electronics
Recycling
Facilities
if

I

1.  Standard Owners. The parties that developed the E-Stewards and R2 standards. Sustainable Electronics
   Recycling International owns the R2 Standard and the Basel Action Networks owns the e-Stewards Standard.
   SERI and BAN play a critical management and support role throughout the certification process. Their
   activities include:

       •   Working with ANAB on oversight of the Standards;
       •   Developing training and guidance materials (e.g. interpretations) to support implementation of the
           Standards;
       •   Training the CBs on the Standards;
       •   Witnessing audits performed by the CBs, and, at times, directly conducting audits (or "spot checks")
           of recycling facilities as part of a quality assurance plan;
       •   Planning for and carrying out updates to their Standards with public comment processes;
       •   Having advisory boards; and
       •   Managing fiscal matters for their Standards.
                                                 11

-------
    It is important to note that both BAN and SERI issued new versions of their Standards in 2013 while the Study
    was underway, and continue to develop comprehensive and effective training and guidance materials to support
    implementation of the Standards. Highlighted below are key changes that were made to the Standards in 2013:

        •   Updating the language in both Standards to:
               o   Enhance language regarding export requirements;
               o   Update operational criteria for environmental, health, and safety;
               o   Provide greater detail of reuse requirements; and
               o   Expand definitions sections.
        •   Updating and improving guidance documents to aid stakeholders in implementing the Standards;
        •   Formalizing operating structures for the legal entities that actually own the Standards, including hiring
           additional staff in both Standards'  organizations to better support the implementation of the Standards;
        •   Approving additional CBs to audit to each Standard;
        •   Updating training programs for CB auditors, electronics recyclers, and other stakeholders, reviewed
           and approved by the Standards owners;  and
        •   Improving quality assurance systems, including programs such as witness audits or spot checks by the
           Standards owners on CB audits and on facilities.

    In order  to  ensure  that  the  new versions of the  Standards  -  and associated  guidance and changes in
    implementation - were appropriately addressed, EPA extended the timeframe to complete the Study to include
    additional stakeholder interviews and audit observations.

2.  ANSI-ASQ National Accreditation Board (ANAB). ANAB is a non-governmental organization that provides
    accreditation services to public- and private-sector organizations and is jointly owned by the American National
    Standards Institute (ANSI)  and the American  Society for  Quality (ASQ). The ANAB portfolio covers the
    spectrum of conformity assessment accreditation to recognized standards, including testing and calibration
    laboratories, inspection, and certification bodies.   ANAB  ensures that Certifying Bodies are operating in
    compliance with standard practices set forth for Certifying Bodies in international standards, such as ISO 17021.
    In the electronics recycling system, ANAB is currently the only accrediting body authorized to determine the
    competency of the Certifying  Bodies responsible for certifying recycling facilities to the R2 and e-Stewards
    standards. To this end, ANAB meets regularly with the Certifying Bodies and conducts regular witness audits
    of Certifying Body auditors. ANAB  participated in the  development of the minimum requirements for
    Certifying Body  competency, although  the  Standards  owners   can  require  further competencies. EPA
    interviewed ANAB representatives during the Study and also  observed two ANAB Witness Audits as part of
    the Study.

3.  Certifying Body. CBs are the organizations responsible for certifying electronics recycling facilities to the e-
    Stewards and R2 standards. CBs hire and train auditors to  certify electronics recycling facilities as meeting (or
    not meeting) the criteria in Standards, and ensure their auditors  are properly trained on the requirements
    associated with the Standards. Most CBs utilize training programs offered by third parties for their initial auditor
    training, complemented by additional in-house training. As of January 2016, six CBs are accredited to certify
    electronics recycling facilities  to one or both of the  Standards:
Table 1. Certifying Bodies and the Standards
Certifying Body
NSF-ISR
Orion Registrar, Inc.
Perry Johnson Registrars, Inc.
SAI Global
SGS
e-Stewards
m
m
m
m

R2
m
m
m
m
m
                                                   12

-------
Certifying Body
TUV SUD America
e-Stewards

R2
m
4.  Auditors. Hired by CBs, auditors perform the audits of the electronics recycling facilities seeking to obtain or
    maintain certification. Auditors are the 'front line' of certification - going on-site and examining all aspects of
    the management system set up by an electronics recycler to ensure it meets the criteria of the Standard(s). EPA
    observed nine audits as they were performed by the CB auditors over the course of the Study.

5.  Electronics recycling facilities implement certified management systems to improve quality and to protect
    environmental and human health and  safety. Electronics recycling facilities can be certified to one or both of
    the Standards. EPA observed nine audits of electronics recycling facilities seeking to obtain or maintain their
    certification to one or both Standards. While the primary focus of the audit observations was to assess how
    effectively the  CB auditors  conducted their audits, EPA also noted how the facilities were  operating in
    conformance with the Standard(s), such as how well they demonstrated knowledge and understanding  of the
    Standards' requirements. EPA also interviewed the recycling facility owners or operators outside of the formal
    audits to gain additional information and hear their perspectives on strengths and opportunities for improvement
    in implementation.
1.2.  Study Purpose and Design
                                                                           1.
                                                                           2.
                                                                           3.
                                                                           4.
   Study Design
Stakeholder Interviews
Audit Observations
Data Analysis
Final Report
The purpose of the Study was to conduct a system-level review of how the two
used  electronics  recycling  certification  programs  in the U.S.  are  being
implemented,  in  order to  determine  if  they are  being  implemented
transparently, consistently, and are achieving the desired results. The  Study
approach was to examine whether the entire system of certification - from the Standards owners, to ANAB, to on-
the-ground CB auditors - is operating to meet its desired results. Accordingly, the objectives for the Study are to
identify areas of strength in implementation and, where appropriate, to make recommendations for improvement.
The Study consisted of interviewing key stakeholders responsible for implementing the Standards, observing audits
of facilities seeking to obtain or maintain their certification, and analyzing the information to develop the findings,
conclusions, and recommendations in this  final report.

Stakeholder Interviews

EPA conducted a total of 39 interviews with stakeholders across the electronics recycling system, including all of
the key entities that are critical to the implementation of the Standards. Interview questions were  different for each
stakeholder group in order to reflect their different roles, responsibilities, and perspectives with regard to the
electronics recycling certification process. The objective of the first 11 interviews, conducted prior to observing
any audits, was to establish baseline information about how the different stakeholders across the system were
implementing the Standards, identify potential issues of concern, and provide an opportunity for the interviewees
to share their perspective on how implementation could be improved. The stakeholders interviewed were:

       Representatives  from  R2 (owned by Sustainable Electronics  Recycling International and e-Stewards
       (owned by the Basel Action Network);
       An ANSI-ASQ National Accreditation Board representative;
    •   A national-level certified facility;
       The six  Certifying  Bodies authorized to certify recycling facilities to the Standards:  NSF-ISR; Orion
       Registrar, Inc.; Perry Johnson Registrars, Inc.; SAI Global; SGS; and TUV SUD America; and
    •   The Electronics Manufacturers Recycling Management Co. (MRM),  which represents a consortium of
       electronics manufacturers.

An  additional 20 interviews were conducted concurrently with the audits that were observed in order to obtain
further insight from the auditors and the facilities  regarding  opportunities to improve implementation of the
                                                   13

-------
Standards. These post-audit interviews were always conducted outside of the formal audit to ensure EPA did not
influence or interfere with the audit. The stakeholders interviewed were: the nine Certifying Body auditors; two
ANAB auditors who were acting in their oversight role witnessing Certifying Body auditors conduct two audits;
and the owners or operators of the nine facilities whose audits were observed as part of the Study.

Finally, eight stakeholders from the initial interviews were re-interviewed in 2014 to  better understand their
perspective on how the implementation of the Standards had changed with the issuance of the new 2013 versions
of the Standards. The stakeholders interviewed were: ANAB, five Certifying Bodies (NSF-ISR, Orion Registrar,
Inc., Perry Johnson Registrars, Inc., SAI Global, and SGS), and the Standards owners (SERI and BAN).

The information shared with EPA as part of the interviews was directly incorporated into the findings of the Study
and is reflected in Section 2 of this report. EPA did not undertake any additional activities to verify the information
that was provided beyond that which was stated or provided by the interviewees.

Audit Observations
EPA observed nine audits of electronics recycling facilities between August 2013 and April 2015. EPA observed
audits performed by five of the six Certifying Bodies (including observing two audits witnessed by ANAB in its
oversight role), which collectively are responsible for auditing over 98% of the facilities that are certified to one or
both of the Standards. Though limited to nine in accordance with Paperwork Reduction Act requirements,8 in terms
of the  different types of audits (e.g., Stage 2,  surveillance, recertification),  facility sizes  (ranging from 5-40
employees;  single  facility vs.  multiple facilities  across  the  country), and  services  offered  (reuse, manual
disassembly, shredding, data destruction, etc.), the audit observations reflect a cross-section of the types of facilities
that can be certified to the Standards. The audits also included an equal number of R2 and e-Stewards audits, as
well as one combined audit where a facility was seeking certification to both Standards, as well as audits to both
versions of the Standards. Five audits were to the 2008 version of the Standards,  and four were to the 2013 versions
of the Standards.

Protocols were  developed and followed before, during, and after the audits, to ensure consistency in how the audits
were observed and the information captured and analyzed. EPA developed  an analytical tool to support the
observations, an Audit Observation Checklist, with 20 topic areas encompassing 144 discrete items, to use to rank
(from 1 to 5) the CB Auditor's effectiveness (see Appendix B of this Report). The 20 topic areas were derived based
on all auditable elements present in both of the Standards. Because the Study period spanned the time when both
Standards were revised, the audit observation checklist was updated mid-Study to reflect  the new versions of the
Standards. The Checklist also included questions to ask of each key participant in the Audit Observation (CB
Auditor; Facility, and, as appropriate, ANAB Witness Auditors), at the end of the audit, to gain their perspectives
about overall effectiveness of the Standards' implementation.

Table 2, below, lists the observed audits in random order, and includes the facility operation type (services offered),
size, and type of the audit observed, including whether it was an ANAB Witness Audit. It  should be noted that for
all audits EPA observed,  the CB ultimately recommended the facilities be certified.
8 The Paperwork Reduction Act was enacted to minimize the paperwork burden for individuals; small businesses; educational
and nonprofit institutions; federal contractors; state, local, and tribal governments; and other persons resulting from the
collection of information by or for the federal government.

                                                    14

-------
  Table 2. Summary of Audits Observed (In Random Order)

1.
2.
3.
4.
5.
6.
7.
8.
9.
Type of Operations
Manual disassembly, reuse and
refurbishment, shredding circuit boards,
Hg-containing component removal,
break/cut Cathode Ray Tubes (CRTs),
baling, data destruction
Manual disassembly, shredding circuit
boards and hard drives (HDs), break/crush
CRTs, baling, data destruction
Manual disassembly, reuse and
refurbishment, shredding HDs, break/cut
CRTs, baling, data destruction
Reuse, data destruction, manual
disassembly
Recycling, manual disassembly, baling
Reuse, refurbishment, recycling, data
destruction, manual disassembly, baling
Recycling, manual disassembly,
shredding, data destruction
Recycling, manual disassembly, data
destruction
Asset recovery and remarketing, recycling,
data destruction
# of Employees and
Type of Company
35; N. American multi-site e-cycler
15; Single-site e-cycler
32; U.S. multi-site e-cycler
5; U.S. multi-site e-cycler
Not included in audit plan
U.S. multi-site e-cycler
Not included in audit plan
Single-site e-cycler
13; Single-site liquid crystal display
(LCD) screen de-manufacturing
specialist
8; Single-site e-cycler
40; Single-site e-cycler
Type of Audit
R2:2008;
Re-certification
R2:2008;
Stage Two
e-Stewards:2008;
Re-certification
e-Stewards: 2008;
Stage Two
(ANAB Witness
audit)
R2and
e-Stewards: 2008;
Surveillance
(ANAB Witness
audit)
R2:2013;
Stage Two
e-Stewards 2.0;
Surveillance
e-Stewards 2.0;
Re-certification
R2:2013;
Surveillance
Data Analysis
Information from  the  interviews and  audits was aggregated and  analyzed to  identify patterns of strengths,
opportunities for improvement, and other trends. With regard to the audit data, average ratings across all nine audits
for each of the 144 discrete elements of the Checklist were calculated, and  those averages were then weighted
according to the number of audits in which each element was  assessed.9 The weighted averages for each of the
elements were then used to calculate an average overall score for each of the 20 Topic Areas (e.g. Health and Safety,
Reuse, etc.) to provide a quantitative sense of how well that particular area of the Standards was audited during the
audits that EPA observed. See Section 3 of this report for a comprehensive description of the data aggregation and
analysis process.

Final Report
This report documents the Study's findings, conclusions, and recommendations, which are based on the interviews,
audits, and an analysis of the patterns and trends identifying strengths and opportunities for improvement.
9 Not every element was assessed in each of the nine audits due to the requirements of different audit types (e.g. Stage 2,
surveillance, and recertification) and of the different versions of the two Standards (2008 and 2013).
                                                   15

-------
1.3.  Study Limitations
Table 3 provides a summary of the key limitations of the Study, which are important to provide context for the
findings and conclusions, as well as for future studies that may be undertaken.

Table 3:  Implementation Study Limitations	
  Limitation
Discussion
  Snapshot in
  time in a rapidly
  changing industry
    Interviews and audit observations were conducted from March 2013 through May
    2015, and only represent findings during that period.
  Subset of entire
  universe of
  electronics recyclers
  studied
    As of December 2015, more than 550 U.S. electronics recycling facilities were
    certified to one or both of the R2 and e-Stewards Standards.10 As such, the
    Study's findings should not be construed to apply to the full universe of
    electronics recyclers operating in the U.S.
  Limited sample size
    The Paperwork Reduction Act limited the number of interviews and audits that
    could  be observed by  the  EPA.11 The Study's findings  are based on  the
    stakeholder interviews EPA conducted and the audits  EPA observed, which reflect
    a small percentage of all certified electronics recycling facilities and  of all auditors
    working in the electronics recycling industry in the U.S.	
  Role of the
  Audit Observers was
  observational only
    EPA did not ask direct questions while observing the Certifying  Body Auditors
    conduct the facility audits. Findings based on the observed audits are therefore
    limited  to what  could  be seen rather than  discussed with the Certifying Body
    auditors.
  Facilities agreed to
  let EPA observe
  audits
    For the nine audits EPA observed, the Certifying Bodies contacted their clients
    (facilities) in advance, and those facilities all agreed to allow EPA to observe their
    audits.  It can be surmised that the clients had confidence in a positive outcome
    of the audit in order to agree to let EPA observe the audit.	
  Not all CBs observed
  in audit portion of
  the Study
    EPA observed audits performed by only five of the six Certifying Bodies during
    the Study. As of the writing of this report, those five Certifying Bodies are
    collectively responsible for auditing over 98% of the facilities that are certified to
    one or both of the Standards.
  No Stage 1 audits
  were observed
    Audit Observations were for facilities obtaining Stage 2 certification, surveillance,
    or re-certification audits; no Stage 1 (readiness review) audits were observed.
    Stage 1 audits primarily consist of reviewing facility documents in preparation for
    a Stage 2 audit, where certification may first be achieved.
10 Total number of certified recyclers as indicated on both the R2 and e-Stewards web sites.
11 The Paperwork Reduction Act was enacted to minimize the paperwork burden for individuals; small businesses;
educational and nonprofit institutions; federal contractors; state, local, and tribal governments; and other persons resulting
from the collection of information by or for the federal government.
                                                   16

-------
2.  Study Findinas
                                                                                   Act
                                                                                 (Area 20)
                                                                                 Check
                                                                                (Areas 15-19)
                                        Plan
                                      (Areas 1-2)
                                        Do
                                     (Areas 3-14)
This section presents the findings from the stakeholder interviews conducted and
audits observed.  Sections 2.1 through 2.20 present the Study's findings
organized under the 20 topic areas that are included in both Standards. The
findings primarily represent information from the nine audit observations with
supporting information from the interviews, as appropriate. Section 2.21
documents any additional findings from the stakeholder interviews that did not
directly fall within one of the 20 topic areas, but are nonetheless critical to
informing the Study's  conclusions and recommendations.

The 20 topic  areas parallel the organization of the Audit Observation Checklist
(Appendix B), which includes 20 key areas and 144 discrete items. The 20 topic areas also follow the Plan-Do-
Check-Act (PDCA)  management  system,  which is the foundation of  all  certified  systems  for control and
continuous improvement processes. The electronics recycling system has roots in ISO 9001 Quality Management
Systems Requirements and,  as a result, PDCA principles are incorporated as part of the certification requirements.
It is also important to note that the PDCA system results in some subject areas being examined multiple times during
an audit. For example, the auditor may be assessing health and safety (H&S) first through the 'Plan' lens ('Was an
H&S hazard analysis conducted so the facility can plan appropriate procedures?'), then 'Do' ('Do employees know
and follow H&S requirements?'), then 'Check' ('What mechanisms are in place to ensure H&S procedures  are
followed and tracked?'), and finally 'Act' ('Is management following up to ensure non-conformances to the H&S
procedures are corrected?').
Key Topic Areas Assessed in the Study

   1.  System requirements
   2.  System planning
   3.  Training and communication
   4.  Operational controls
   5.  Health and safety
   6.  Reuse
   7.  Data security/data destruction
   8.  Management  of  Focus  Materials  (R2)  and
       Hazardous  Electronic  Waste  and  Problematic
       Components and Materials (e-Stewards)
   9.  Emergency preparedness and response
   10. Materials recovery and disposition
11.  Export  restrictions for  Focus Materials  (R2) and
    Hazardous Electronic Waste (e-Stewards)
12.  Site closure and insurance
13.  Downstream accountability
14.  Monitoring and measurement
15.  Tracking
16.  Evaluation of compliance
17.  Nonconformity, corrective and preventive action
18.  Control of records
19.  Internal audit
20.  Management review
2.1.  System Requirements
Description. The "scope" statement that appears on the certificate issued to an electronics recycler by the Certifying
Body delineates the limit of the functional area(s) within the organization to which certification applies. It is very
important that scope statements be written to align with the requirements of the standard to which the facility is
being certified and to  clearly describe operational systems  (and the related management oversight) that is being
certified. It is one of the first things an auditor should verify.

Strengths.   Verification of  this  area by auditors was done  consistently. Because  this element of conformity
verification comes from the ISO standards, it is done fairly uniformly by all auditors. One auditor actually stopped
the audit  when a discrepancy was  discovered between the  scope statement and actual practices,  and sought
clarification from the CB's management, resulting in a quick resolution of the matter.
                                                   17

-------
Opportunities for Improvement.  Generally conforming practices were observed with regard to this system-level
requirement. However, one facility included 'refurbishing' in its R2 scope statement, when no refurbishing activities
were taking place. Further, this facility's scope statement did not include "sorting, collection, dismantling, and data
destruction" although those activities were taking place. The auditor did not inquire about the discrepancy between
the scope and the activities on site - for example, to determine if refurbishing was an activity that was planned to
be added, occurred at another facility, or had been discontinued; or why the scope statement was not in alignment
with actual practices.

Key Takeaways.  It is important that both facilities and CB auditors understand the importance of aligning scope
statements  with company operations.
 (2.1) SYSTEM REQUIREMENTS
 ISSUE
SUGGESTED STRATEGY FOR IMPROVEMENT
 Aligning scope statement in certificate
 with company actions
•   Ensure  this  high-level  item is  reviewed in  seminars and  educational
    settings instructing e-cyclers and  auditors about certification to ensure
    relevant aspects of company operations are audited effectively.
2.2. System Planning
Description. System planning requirements involve identifying legal requirements, understanding health and safety
risks, having a document control procedure and setting objectives and targets. System Planning is a foundational
part of the 'Plan' phase in a Plan-Do-Check-Act (PDCA) management system.

Strengths.  Thorough auditing practices were observed, including instances in which auditors:

    •   Verified Policy/Plan based on hierarchy of responsible management;
    •   Reviewed system documentation; and
    •   Reviewed procedures for identifying legal and other requirements.


Opportunities for Improvement. Generally conforming practices were observed with regard to this system-level
requirement. Generally CB auditors' understanding of system planning appeared  to be well developed. The
exception to this is in the area of auditor awareness of legal requirements. Attention paid to this element by auditors
was not consistent with its importance. Several auditors failed to request and review documents proving the facility
was tracking and ensuring  its own compliance with all legal requirements, especially with regard to both data
destruction and export. In addition, management and employee awareness of legal risks was not always audited.

The observations  also indicated that opportunities for improvement on the part of recycling facilities exist in the
following areas:

    •   Comprehensive identification  of legal  requirements - particularly pertaining to critical  areas including
        equipment operation (such as shredders), export, air emissions, and data security;
    •   The complexity and ever-changing nature of legal and export requirements poses a challenge to a facility' s
        ability to effectively create a system to ensure compliance with applicable laws and regulations;
    •   Evaluation of environmental health and safety risks - these were conducted, but in a number of the audits
        observed  this analysis was not comprehensive,  failing to address key issues  such as potential release of
        toxins or dust exposure;
    •   Communication and awareness of roles and responsibilities  was not always clear. For example, in one case
        responsibilities were managed at the corporate level with no responsible party  at the facility level; and
                                                   18

-------
    •  Ensuring use of the most current documents and procedures in various day-to-day operations to avoid
       variances between Procedures in use and what is listed on the Master Document List.

Key Takeaways. Clear identification of legal and other requirements and environmental health and safety risks is
fundamental for an effective environmental, health and safety management system.  Suggested actions include the
creation of tools and methodologies to expand knowledge in this area for all system participant. An example would
be a succinct summary of all federal laws pertaining to  e-waste. In addition, many interviewees and facilities
suggested providing additional guidance and clarification on the CRT rule.
(2.2) SYSTEM PLANNING
ISSUE
Identification of
legal and other
requirements
CRT Rule
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Create a database for auditors identifying key legal requirements for each of the jurisdictions in
which the CB operates. The database could be designed to generate state-specific checklists
prior to an audit. Auditors need related guidance regarding expectations for the level of detail and
comprehensiveness.
• Provide a clear summary of federal laws pertaining to e-Waste processing, management,
transportation, and export; conduct webinars on legal requirements.
• Enhance evaluation of auditor competency regarding knowledge of legal and other requirements,
as part of auditor training requirements.
• Develop outreach and training materials to clearly explain the CRT rule.
2.3. Training and  Communication
Description. Part of the 'Do' phase of the Plan-Do-Check-Act system is establishing roles and responsibilities, and
ensuring resources are in  place for system  implementation through  training  and  external  and internal
communications. Additionally, training and communication at a certified facility is the  means by  which all
employees and stakeholders are made aware of the requirements and procedures of the environmental health and
safety management system, including all electronics recycling-specific requirements from the Standards.

Strengths. Under this area, EPA did identify several good examples of thorough auditing by auditors.  For example:
    •   Using  auditing of  this element to identify facility-specific training and communication opportunities in
        important  areas (e.g., forklift maintenance; failure to train temporary employee  in up-to-date work
        instructions);
    •   Asking to  see facility's customer  complaint log, then following the audit trail by checking the Non-
        Conformance record for a complaint involving a lack of personal protective equipment (PPE) at a customer
        facility, contrary to agreement; auditor verified that the complaint was closed appropriately;
    •   Identifying the lack of U.S. Department of Transportation  (DOT)/Hazmat  training as a minor Non-
        Conformity; and
    •   Confirming that a corrective action led to a change in procedure.

For facilities, the following strengths were noted:
    •   Applying the principles of the Standard to which they are certified, to develop easy-to-remember  reminders
        and training tools very specific to their laborers' level of understanding - and then supporting employees'
        learning through positive reinforcement; and
    •   Improving  training to help prevent shipping to non-approved downstream vendors (i.e., vendors  not vetted
        to the standard requirements).
                                                  19

-------
Opportunities for Improvement. Conforming practices were generally observed with regard to this system-level
requirement. However, opportunities for improvement related to training and communication for facilities were
indicated in areas such as:

    •  Training employees on environmental aspects and health and safety hazards;
    •  Creating written procedures for training temporary employees;
    •  Demonstrating that key employees understand hazardous content (i.e., substance), by component;
    •  Demonstrating that key employees understand the definition of Universal Waste and applicable federal
       and state regulations (including battery storage requirements);
    •  Demonstrating necessary comprehensive knowledge of import/export and legal compliance matters;
    •  Demonstrating knowledge and awareness of fire evacuation and emergency preparedness procedures;
    •  Ensuring understanding of the management system requirements across a work force that may not use
       English fluently; and
    •  Communicating awareness of roles  and  responsibilities clearly.  In one case, responsibilities that were
       corporate in nature were not being effectively translated at the facility level.

For auditors, opportunities for improvement were identified as follows:

    •  Asking for training materials to review them for adequacy;
    •  Auditing how employee communication occurs;
    •  Looking at a facility's processes for reviewing and responding to customer complaints, as an indicator of
       conforming communication practices;
    •  Identifying facility staff that need to be better trained on legal, downstream material flows, and related
       issues;
    •  Ensuring this element is audited (one auditor did not appear to audit this element); and
    •  Training in auditing for hazardous waste and export requirements.

In addition, findings  from the  interviews and audits indicated there is a system-wide lack of comprehensive
knowledge on the environmental, health and safety (EH&S) legal compliance requirements  specific  to  the
electronics recycling industry. For example, facilities were  observed that appeared to not understand Universal
Waste requirements for battery management. This lack of understanding may be attributed to inadequate training
and communication with regard to the applicable requirements in this unique  industry. All stakeholders (CBs,
auditors, ANAB, facilities, regulators, and Standards owners) demonstrated limitations of knowledge in this area
during the Study. It appears that there is not sufficient access to both initial and ongoing training in the many aspects
of electronics-recycling-specific compliance. Although improvements were observed during the Study period,
particularly with the issuance of the new versions of the e-Stewards and R2 standards with their emphasis on
training, these knowledge limitations are still problematic and present significant opportunities for improvement.

Further, the knowledge base in the industry is still North American-centric, even though electronics recycling
operates in a global commodities market. As U.S.-based stakeholders in the  electronics recycling industry expand
beyond the U.S., this  insufficient comprehensive knowledge is further compounded, especially in the context of
international trade laws and conventions. This may be due to inadequate access to authoritative knowledge of legal
requirements related to EH&S, data destruction, and export.

Key Takeaways.  Training  and communication are fundamental to the success of these standards. Though generally
audited well, there is an opportunity for auditors to better verify if facility employees, internal auditors, and other
stakeholders in the management system have adequate training to thoroughly support a facility's use of training and
communication tools to implement its certified EH&S management systems. Effective training is essential and is
needed on virtually all the  auditable subject-matter areas listed in this report.
                                                   20

-------
Suggested strategies for improvement include exploring and enhancing training tools covering the basics of health
and  safety,  operational controls,  data security and destruction, emergency response and preparedness, legal
requirements, management of focus materials (FMs). hazardous electronic wastes (HEWs). and problematic
components and materials (PCMs). This training could be implemented in a consistent and accountable manner
(e.g., continuing education units (CEU) style). In addition, such training  might provide opportunities for private
sector companies to deliver effective education, much as is done in health care, legal, and other professions where
CEUs or training updates are needed to maintain licensing or, in the electronics recycling industry, certification.
(2.3) TRAINING AND COMMUNICATION
ISSUE
Training and
communication
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Develop best practices or standardized, easy-to-follow training
industry stakeholders (CBs, recycling facilities) that covers the
operational controls, data security and destruction, emergency
legal requirements, management of FMs, HEWs and PCMs, etc.
tools for electronics recycling
basics of health and safety,
response and preparedness,
• Provide consistent and 'certified' (i.e., Continuing Education Units (CEU)-style) training covering
health and safety, operational controls, data security and destruction, emergency response and
preparedness, legal requirements, management of FMs, HEWs and PCMs, etc.
2.4.   Operational Controls
Description. Operational control pertains to the processes and procedures that facilities set up and use to ensure
they maintain control over the significant environmental,  health and safety aspects of their operations.  It
encompasses setting up operating procedures that spell out how operating criteria  will be  determined and
established.  Examples include  detailed  instructions for processes  such as  waste management and storage of
chemicals; procedures to evaluate the need for engineering, administrative and personal protective equipment (PPE)
controls; communications mechanisms; and change management. As part of the 'Do' stage of Plan-Do-Check-Act,
operational controls ensure an effective EH&S management system.

Strengths.  Because operational controls are required by the Standards, certified facilities develop processes and
procedures tailored specifically to the facility and operations. The  monitoring  and measurement of operational
controls  (e.g., using inspection checklists) ensures the environmental health  and safety management system
(EHSMS) is not shelved until the next audit year. In addition, one Study interviewee mentioned use of PPE vending
machines as being an effective tool in getting better technician compliance in using PPE regularly.
Opportunities for Improvement.
included:
Opportunities for improvement noted in the Audit Observations for auditors
    •  The need to consistently verify that written procedures were implemented correctly by staff. For example,
       auditors missed important audit opportunities such as:
       o  Reviewing the Focus Material (FM) Management Plan - and following up to verify that FMs such as
           mercury-containing devices, CRTs,  batteries, etc., were being  managed according to the stated
           procedures in the warehouse and in associated documentation.
       o  Following the audit trail to confirm the extent to which engineering or administrative controls were
           used for specific areas ranked with a high health and safety (H&S) risk.
       o  Asking technicians about engineering controls  or other  required  procedures or not noting that
           technicians did not appear to be clear about some procedures.
       o  The need to ensure that practices observed during the audit (e.g., large proportion of CRTs broken in
           gaylords, poor ergonomic controls, failure to use PPE) which were at a variance from controls stated in
           written procedures were addressed in audit findings.
                                                  21

-------
    •  Ensuring review of purchased materials impacting H&S (e.g., information from propane suppliers).
    •  Auditing a facility's Management of Change Procedure, which is a standard operating procedure to be
       followed when change is planned for  a facility (e.g., when  considering adding a shredder, management
       should complete the Management of Change Procedure to ensure proper control of operations in accordance
       with the requirements of the specific Standard the facility is certified to, including regulatory requirements).

Opportunities for improvement for the electronics recycling facilities included:

    •  Translating risks identified in risk and hazard assessments into procedures, control requirements, and areas
       where more information is needed to manage risk correctly. For example, facilities were observed:
       o  Ranking battery explosion as a 'high' H&S risk but not requiring PPE when working with batteries;
       o  Lacking a procedure for Universal Waste (UW) storage nor associated controls for risks associated with
           UW storage, disassembly, transportation, etc.; and
       o  Identifying hearing loss  as a risk, with hearing protection as a control, but optional for technicians -
           but then failing to follow up and monitor if hearing loss was occurring.
    •  Ensuring proper use of PPE and controls. EPA observed, for example, employees with no arm protection,
       wearing masks improperly, no eye protection and no ear protection - in operating situations which would
       necessitate such PPE. EPA also observed missing machine guarding in a shredder area.

Key Takeaways. Note that in the interviews conducted directly after the audits, several interviewees pointed to the
problems electronics recyclers have in completing  adequate and  accurate risk and hazard assessments.  The
interviewees suggested that electronics recyclers  do not  recognize the importance of this  step, how  to do it
accurately, and how it should be translated into procedures, controls,  follow-up activity, and monitoring. These
operational control tools and systems, when well-used and well-monitored by key staff and management, form one
of the foundations of a good EH&S management system.  Recommendations include the need to build better
understanding system-wide on how to do risk and hazard assessments and how to use operational controls.
(2.4) EFFECTIVE USE OF OPERATIONAL CONTROLS
ISSUE
Operational
controls
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Use training tools and best practice case studies to build better understanding
how to do risk and hazard assessments and how to use operational controls.
• Use training tools to build better understanding for auditors on how to effectively
operational controls are being used correctly.
for facilities of
audit whether
2.5.   Health and Safer
Description. Addressing Health and Safety (H&S) requirements is one of the most important and complex aspects
of being certified to the e-Stewards and R2 Standards. Health and safety is a critical 'Do' phase of the Plan-Do-
Check-Act system, which encompasses:

    •   Knowing and meeting applicable health and safety laws;
    •   Understanding all H&S hazards posed by working with end-of-life electronic equipment;
    •   Using appropriate engineering and administrative controls;
    •   Ensuring correct use of personal protective equipment;
    •   Maintaining workplace hygiene;
    •   Knowing and managing H&S risks;
    •   Monitoring;
    •   Conducting programs, training and communication about H&S, and
                                                  22

-------
    •   Completing extensive documentation of all H&S procedures and practices.

Strengths. Findings from the interviews and audits indicate that because of the Standards, electronics recycling
facilities have implemented significant steps toward improving industry health and safety, such as using ongoing
health and safety committees, air quality testing, medical surveillance, and hazard assessments. Findings indicate
that these practices were minimally implemented or less common prior to the emergence of standards specific to
the electronics recycling industry. Adding OHSAS  18001 (referenced in each Standards' second iteration) is further
raising the bar and encouraging better health and safety practices.

Standards owners are aware of and are addressing these issues. E-Stewards noted that the 2013 National Institute
for Occupational Safety and Health (NIOSH) report on lead (Pb) issues at electronics recycling facilities12 came out
immediately before version 2.0 of the e-Stewards Standard was published. E-Stewards plans to specifically address
the need for better monitoring (including possible bio-monitoring of workers, and surface sampling) and reporting
of Pb levels, via its Sanctioned Interpretations, leading to best practices for e-Stewards.13  In addition, SERI has
developed its modular Implementation Guide for R2-certified electronics recyclers; it includes a module addressing
Provision 4 of the R2:2013 Standard addressing health  and safety issues.14  NIOSH's ongoing research data is
beginning to contribute better data and baselines for  Pb-related or other industrial hygiene issues for electronics
recyclers, (see NIOSH's July and September 2014  articles15).

Opportunities for Improvement. Conformity with this area was one of the most difficult for the observed facilities
and posed challenges for all other players in the system. Opportunities to better audit health and safety for auditors
included:
    •   Ensuring, through records review and asking  employees questions, that all workers (temporary as well as
        permanent) are aware of and in conformity with the H&S system requirements;
    •   Reviewing the PPE or Hazard Communication Procedure, the Illness and Injury Prevention Program, and
        ensuring conformity in practice;
    •   Viewing all pertinent records - not just looking at procedures, but asking to view items including:
        o   H&S meeting minutes;
        o   Health-related test results (e.g., lead) or hearing tests, including documentation of informing workers
            of test results;
        o   Monthly and OSHA inspection records;
        o   Location and proof of use of SDSs (Safety Data Sheets), and
        o   Evidence of following workplace hygiene/cleaning procedures;
    •   Demonstrating their understanding of exposures and risk-based controls by asking the facility questions to
        elicit operator knowledge of those issues; and
    •   In one case, an auditor failed to audit the basics of a health and safety program.

Opportunities for improvement for the electronics recy cling facilities included the need for:
    •   Better understanding of hazards analysis, potential exposures, and subsequent use of risk-based controls,
        including taking steps such as:
        o   Creating and actively using procedures based on hazards analysis findings,
12 Letter to Sarah Westervelt, BAN, from Elena H. Page, MD, MPH, Medical Officer and Diana Ceballos, PhD, CIH, Industrial Hygienist;
Hazard Evaluations and Technical Assistance Branch; Division of Surveillance, Hazard Evaluations and Field Studies; National Institute
for Occupational Safety and Health, July 23, 2013, available at www.ban.org/files/NIOSHI Letter  to BAN.pdf
13 Personal communications, Sarah Westervelt, BAN, June 8, 2014, November 6, 2015.
14 See https://sustainableelectronics.org/implementation-guide-provision-4 , accessed November 6, 2015.
15 See Occupational Exposures at Electronic Scrap Recycling Facilities, September 30, 2014, by Diana Ceballos, PhD, MS, CIH and Elena
Page, MD, MPH, http://blogs.cdc.gov/niosh-science-blog/2014/09/30/escrap/ ; also see Evaluation of Occupational Exposures at an
Electronic Scrap Recycling Facility, Report No. 2012-0100-3217, July 2014, by Diana Ceballos, PhD, MS, CIH; Lilia Chen, MS, CIH;
Elena Page, MD, MPH; Alan Echt, MPH, CIH; Aalok Oza, MPH; and Jessica Ramsey, MS, CPE.
www.cdc.gov/niosh/hhe/reports/pdfs/2012-0100-3217.pdf accessed November 6, 2015

                                                     23

-------
        o  Using PPE hazard assessments in alignment with hazards identification,
        o  Improving understanding of SDS and how and when to use them and where to get them,
        o  Knowing and posting occupational health providers,
        o  Conducting air monitoring to determine if medical surveillance is needed, and
        o  Using their management systems to communicate all aspects of EH&S monitoring to those impacted;

    •   Consistent ability to identify hazardous substances  in equipment (e.g., an auditor identified medical
        equipment not labeled as hazardous and also observed leaking batteries, which the facility operator had not
        noticed). Facilities face challenges in this area such as:
        o  Lack of documented information on hazardous substances associated with equipment being received,
        o  Manufacturers  of electronic  equipment not providing information on  hazardous  or  potentially
           hazardous materials in components, and
        o  Understanding risks posed by lead (Pb)  and other toxins in electronic equipment depending on type of
           processing being used;
    •   Better understanding by electronics recyclers of the importance of consistent workplace hygiene procedures
        (housekeeping and cleanliness);
    •   Using standing H&S Committee meetings to further  conformance with the H&S System throughout the
        facility; and
    •   Evaluation of business models that may undermine attention to H&S.  For example, it appears  that paying
        employees on a pound-per-hour basis creates disincentives for workers and managers to adequately attend
        to housekeeping, safety, and ergonomics.

Key Takeaways. This area needs significant attention  from all  stakeholders in the electronics recycling industry
and beyond (from manufacturers through  regulators).  For the  near-term, suggested strategies for improvement
include the need for additional time during audits for  auditors to more thoroughly audit health and safety monitoring,
measurement, record-keeping, and communication issues to give facilities critical feedback on their H&S systems.
Also, electronics recyclers need further training and guidance on conducting proper risk assessments and hazards
analyses, complying with health and  safety requirements,  and using appropriate controls. Additional research is
needed into the potential health hazards of exposure  to electronics in the various ways it is managed at end of life -
related to both short- and longer-term exposures and manifestations.
(2.5) HEALTH AND SAFETY
ISSUE
Health and safety
conformity and
compliance
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Improve compliance with and enforcement of occupational
requirements.
health and safety laws and
• Develop training and guidance for facilities on conducting comprehensive risk assessments and
hazards analyses; use of safety data sheets; applicable legal requirements; monitoring results;
machine guarding and lockout/tag-out; etc.
• Develop ongoing mechanism to identify hazardous substances
evolve.
in electronic devices as they
• Provide training on transportation requirements for hazardous substances (e.g., FAA restrictions
on lithium-ion batteries in airtransport).
• Publish research-based baseline requirements for Pb and other
human health and indoor environment, specific to the electronics
• Provide training and guidance for facilities and auditors on
measurement, recordkeeping, and communication.
toxic substances sampling for
recycling industry.
industrial health monitoring,
                                                   24

-------
2.6. Reuse
Description. As part of the 'Do' phase of Plan-Do-Check-Act, reuse of electronics is encouraged by both Standards
over recycling or disposal. Reusable electronics and components must be tested for functionality, cleared of any
customer data, and packaged adequately for protection during shipment. Outsourcing of testing is addressed along
with management of scrap from the test and refurbishment process. This area is of special concern in the Standards
due to the risk of having equipment and components  destined for reuse being exported to destinations allowing
illegal dumping and other unethical practices.

Strengths.   Good practices  observed included the  use of protective  packaging in shipping, and adequate data
destruction of equipment going for resale. Further, the updated versions of both Standards provide greater clarity in
the Reuse area, as does additional information provided from the Standards to their certified facilities.16

Opportunities for Improvement. A mix of conforming and non-conforming practices were observed with regard
to this requirement.  The Study found that both facilities and auditors do not consistently understand the Standards'
reuse requirements. Opportunities for improvement were noted in the following areas:
    •   Increasing the audit time for this provision;
    •   Better auditing of provisions in both Standards which prohibit sale or donation for reuse  if contrary to
        commercial agreements;
    •   Verifying that equipment is destined for Reuse and not Recycling or Final Disposal; and
    •   Implementing and auditing functionality testing for equipment and components in a consistent manner.
        Specifically, the following issues were noted related to functionality testing:
        o  Several recyclers  outsourced  testing; however there was minimal verification by the recycler  that
           downstream vendors were actually conducting functionality testing;
        o  In several cases, auditors did not request functionality testing results or records and/or recyclers did not
           maintain these records;
        o  A lack of consistency in testing procedures.  For example, in one instance, power-on testing only  was
           performed which was not identified by the auditor as a non-conformance;
        o  Inconsistent interpretation of key terms by auditors and facilities.  For example, definitions of "key"
           functions and "full" functionality testing in the Standards were not consistently interpreted; and
        o  Facilities may be exporting non-functional focus materials without proper documentation of
           functionality testing, which is not being caught by facilities or auditors.

Key  Takeaways.  Consistent definitions of required functionality testing in the Standards or related Guidance
documents are essential to ensure effective reusable equipment and to prevent exports of non-tested/non-working
components and/or equipment  - as well as to provide clarification for auditors. Sufficient audit time is also needed
to adequately audit this element, as  well  as tools for auditors to aid  auditing  the reuse function in a consistent
manner.
16 E.g., SERI's March 2015 newsletter, R2 Update, contains an article "Ensuring your sales on eBay conform to R2 testing
and resale requirements," which addresses an area of confusion for electronics recyclers, regarding Reuse.

                                                   25

-------
(2.6) REUSE
ISSUE
Functionality
testing
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Provide better information/tools to ensure export of equipment for reuse and refurbishment complies with
the Standards.
• Develop lists with clear definitions of various functionality tests
facilities and auditors.
and key component testing to assist
• Provide an improved definition of acceptable test methods, more clarification as to what is acceptable,
and additional guidance on different types of testing (e.g., scenarios, flowcharts, or guidelines of good
practices).
2.7. Data Security/Data destruction
Description. This element of the e-Stewards and R2 standards goes beyond baseline requirements of an EH&S
management system and is necessary to protect sensitive information found in data-bearing devices from theft or
misuse. This element is in the 'Do' phase of the Plan-Do-Check-Act system.

Strengths.  The Standards have improved the detail and requirements for this area in the new versions. Very good
auditing of this element was observed during Audit Observations.

Opportunities for Improvement.  Generally conforming practices were observed with regard to this requirement.
Opportunities for improving the understanding of the following areas for both auditors and facilities and other
stakeholders, exist for:

    •  Customer requirements which should drive the level of security and data destruction;
    •  Whether a contract  or other similar agreement is required with downstream vendors performing data
       destruction, and specific language that would prove conformity;
    •  Data  destruction concerns  for non-computing equipment  including  non-smart cell phones,  printers,
       scanners, etc.;
    •  The extension of a facility's  insurance policy to include coverage for data breach issues;
    •  Independent validation of data destruction (third-party or in-house QC function);
    •  How asset tag removal and management are handled;
    •  Data security protection procedures between time of collection and time of hard drive destruction (e.g.,
       theft risks); and
    •  Security at site entrance, facility entrance, warehouse exit/entry points, and security cage.

Key Takeaways. Overall, this is an area of strength.  It is particularly difficult to recommend a "one-size-fits-all"
solution for data security and destruction as electronics recyclers deal with vastly differing levels of security needs
- from those accepting non-data bearing equipment to those handling data-bearing devices with the highest security
clearances needed.
(2.7) DATA SECURITY/ DATA DESTRUCTION
ISSUE
Protection of data
from theft or loss
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Develop training on risks of data theft or loss during transportation,
logistics and drivers-for-hire.
especially with
outsourced
• Develop guidance and training on risks associated with data devices being stored onsite,
including best practices for facility-level security.
                                                  26

-------
2.8.  Management of  Focus  Materials,  Hazardous  Electronic  Waste,  and
Problematic Components and Materials
Description. Both Standards include requirements for responsible management of electronic waste with a particular
attention to equipment that may be hazardous. R2 defines this equipment as Focus Materials (FM), while e-Stewards
defines two categories: Hazardous Electronic Waste (HEW) and Problematic Components and Materials (PCMs).
Each Standard provides a general principle that management of these materials must be done in a manner that is
protective of human health and the environment - although the approach taken to specifying controls varies between
the Standards. This management requirement is part of 'Do' phase of the Plan-Do-Check-Act system.

Strengths.  Good practices observed included instances where auditors focused on: (i) verifying that a facility would
seek to learn what hazardous substances were in non-standard electronic equipment it received, and (ii) reviewing
Universal Waste storage  and labeling requirements. Facilities were observed effectively planning  for and then
correctly separating and storing FMs/HEWs/PCMs.

Opportunities for Improvement. While it  was clear in a  number  of audits observed that there  was good
understanding of appropriate management practices for FMs, HEW, and PCMs, it appeared that some significant
instances  of unsafe  or  inappropriate  practices  were not  identified  by  the  auditors.    For  example:

    •  In  some cases, auditors would verify good management practices for some high-risk areas, but paid
       insufficient attention to others (e.g., safe removal and storage of mercury lamps, CRT breakage,  and
       batteries);
    •  In one facility, it was noted that there was a conjoined warehouse containing large storage units, owned by
       the parent company  to the  certified recycler. The auditor did not ask for records of what was in the
       conjoined warehouse to make sure there were no FMs, HEW, or PCMs stored there, nor asked to physically
       inspect the storage units;
    •  In some cases, it appeared that there was limited verification of labeling and storage time limits for FMs,
       HEW, and PCMs;
    •  There was limited  review  of  notification  and labeling  requirements  for  overseas  shipping  of
       FMs/HEWs/PCMs to precious metals recovery; and
    •  In nearly all cases, verification of labeling and export status for CRTs was not observed.

A number of opportunities for improvements were identified to enhance the operational knowledge of certified
facilities, including better understanding of:

    •  Hazardous content (i.e., substance), by component and/or equipment type;
    •  Appropriate management practices for equipment with hazardous content; and
    •  Definition of Universal Waste and applicable federal and state regulations, including battery  storage
       requirements.

Key Takeaway
-------
health-and-safety perspective. The difficulties electronics recyclers face when trying to obtain information about
hazardous or problematic materials found in equipment and components from manufacturers further compounds
this problem. Inadequate audit time negatively impacts this element, particularly for integrated audits.18

Increased training for appropriate identification and management of hazardous components and materials will begin
to address this problem. Based on experience and the findings of this Study, making information on  hazardous
substances found in equipment available to electronics recyclers, by manufacturers from component-level Bills of
Materials, would further assist in this area.
(2.8) MANAGEMENT OF Focus MATERIALS, HAZARDOUS ELECTRONIC WASTES, AND PROBLEMATIC
COMPONENTS AND MATERIALS
ISSUE
Awareness of
appropriate
management
practices for
hazardous
components
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Encourage all electronic equipment manufacturers to disclose hazardous constituents at the bill-
of-materials level to electronics recyclers, to best protect human health and environment.
• Conduct research into and create incentives for solutions for safe removal of hazardous
components. Could include developing training/webinars, including best practice case study
materials for each type of hazardous material and component, to support appropriate
management of hazardous constituents.
• Better attention to safe dismantling of liquid crystal displays (LCDs). The potential for breaking
the small, fragile, mercury-bearing tubes is high.
• Conduct or support research into protocols for emerging toxins potentially found in end-of-life
electronics that may be released by various processing technologies (e.g., brominated flame
retardant (BFR) emissions from shredding and pulverizing plastics). Support transfer of this
knowledge to electronics recyclers.
2.9. Emergency Preparedness and Response
Description. Each of the  Standards includes requirements for identifying potential emergency  situations and
requires plans for responding to  emergency situations in order to protect worker health and safety and the
environment. Emergency preparedness plans should be tested on a regular basis and reviewed after an emergency
situation.  This is part of the 'Do' stage of the Plan-Do-Check-Act system.

Strengths. Good practices observed, including instances in which auditors:

    •  Reviewed emergency response plans and emergency equipment such as spill kits, fire extinguishers and
       emergency lighting;
    •  Interviewed employees for knowledge of what to do in case of specific emergencies, including  severe
       weather shelters and meeting places for evacuation; and
    •  Issued a nonconformance for not updating and revising procedures  when an emergency drill revealed the
       existing method was not effective (one department could not hear the alarm).

Opportunities for Improvement. Generally conforming practices were observed with regard to this requirement,
including facilities' attention to safe storage  and  shipping of flammable materials such as batteries. However, the
observations indicated that opportunities for improvement exist for facilities as follows:
18 Multiple standards being audited at the same time. For example, a facility may be audited to R2:2013, e-Stewards v2.0,
ISO 14001, OHSAS  18001, and ISO 9001, during the same audit. This can both save time and make the audit much more
complex for the CB auditor. Facilities choose to have multiple certifications for many reasons, including to: meet
requirements of the standards themselves, meet customer requirements, and/or optimize operations.
                                                  28

-------
    •  Understanding response procedures for material-specific spills (e.g., CRT glass, mercury);
    •  Expanding emergency drills (test more than just fire drills - test for severe weather, mock medical, etc.);
       and
    •  Developing safer dismantling procedures to avoid spills.

Generally, auditor understanding of emergency preparedness and response appeared to be well developed; however
in several observed audits, auditors asked about emergency drills, but did not request records as evidence.

Key Takeaways.   Observations and interviews  both  indicated that emergency preparedness and  response is
understood and implemented by facilities and audited effectively by auditors.
(2.9) EMERGENCY PREPAREDNESS AND RESPONSE
ISSUE
Awareness of
appropriate
management
practices for
emergency
preparedness
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Creation of best practice guidelines for material-specific spills (e.g., CRT glass and mercury
spills) and educate employees on the dangers of personal exposure to lead and mercury.
• Expanded emergency drill testing. The majority of facilities focus on just fire drills every year.
Testing could be rotated to include mock medical emergency, mock spill, severe weather, etc.
2.10.  Materials Recovery and Disposition
Description. Each of the Standards includes requirements for acceptable processing, recovery, or treatment of
FMs/HEWs/PCMs materials, including that all operations are fully licensed and permitted. This is also a part of the
'Do' phase of the Plan-Do-Check-Act system, and is unique to the electronics recycling Standards.

Strengths. Good practices were also observed, including instances in which auditors:

    •   Selected specific FMs/HEWs and followed them through to final  disposition, verifying volumes and
        acceptable destinations;
    •   Verified that the recyclers were themselves verifying that downstream processors were meeting the
        requirements to control downstream destinations, particularly when the audit function was outsourced;
    •   Focused on high-risk areas such as battery storage; and
    •   Verified that transporters had the necessary regulatory authorizations and no significant violations.

Opportunities for  Improvement. Generally conforming practices  were observed with regard  to this important
requirement. However, the observations indicated that opportunities for improvement exist in the following areas
for auditors:

    •   Comprehensive  verification of  written procedures and associated  controls for ensuring downstream
        processors conformed to Standards' requirements were not audited in  about half of the audits observed;
    •   Consistent review of final disposition (destinations) and processing of residuals arising from the hazardous
        constituents of electronic waste (including halogenated compounds from plastics or resin materials); and
    •   Sufficient verification of controls required (which differ by standard) prior to shipment throughout the
        recycling chain; for example:
        o  R2  requires  identification of countries  receiving such  shipments  and obtaining documentation
           demonstrating that each country legally accepts such shipments.  A facility can then only export to
           countries for which it has the required documentation as well as controls for downstream processing;
           parties in the recycling chain must meet R2's reuse and  contractual requirements.
                                                  29

-------
       o  E-Stewards requires prior approval to confirm  export, reuse, materials recovery and disposition,
           contractual, and downstream accountability requirements are met.

For facilities, it appeared that attention to safe packaging of hazardous equipment (e.g., CRTs, batteries, etc.)
appropriate to protect public health and environmental during transportation, as required by federal law19, could be
improved.

Key  Takeaways.  It  appeared that there may be a significant opportunity for improvement in verification of
downstream materials recovery and  disposal through to final disposition. Observations and interviews both
indicated that managing downstream  conformity to the Standards is perhaps the most challenging aspect of the
Standards.  In this  small sample of observations, it  appeared that there may be a significant opportunity for
improvement in the methods and level to which audit and other records are reviewed by auditors to verify the
efficacy of controls for downstream materials recovery and disposal through to final disposition. Adequate audit
time and sufficient tools are suggested strategies for improvement.
(2.10) MATERIALS RECOVERY AND DISPOSITION
ISSUE
Awareness of best
practices
Verification of
downstream
processors
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Improvement of training materials for auditor knowledge of downstream processing options and
best practices.
• Update relevant web sites, deliver webinars and/or email updates on tools and guidance to help
electronics industry stakeholders improve their understanding of best practices.
• Provision of auditing tools/ training/ questions to assist auditors to fully explore audit trails for
downstream processing.
• Creation of case studies for training materials on how to audit downstream processing.
2.11.  Export restrictions for Focus Materials and Hazardous Electronic
Waste
Description. Export controls, for Focus Materials and Hazardous Electronic Waste (FMs and HEWs), are required
in both Standards with provisions to ensure export only occurs to countries that legally accept the shipped items or
materials in conformance with applicable national laws and/or international laws, such as the Basel Convention
for the Transboundary Movement of Hazardous Waste or OECD Decision C (2001)107/FINAL. The e-Stewards
Standard includes requirements to limit the shipment of hazardous waste to countries that are not members of the
OECD/EU consistent with the Ban Amendment under the Basel Convention.  The Ban Amendment is not in force,
but has been implemented by some Parties to the Basel Convention. This is also a part of the 'Do' phase of the
Plan-Do-Check-Act system that is unique to the electronics recycling Standards.

Strengths. The Standards' second versions (issued in 2013) both made their provisions regarding export more
specific, which has further highlighted the need in the entire system for attention to responsible export of electronics.
Prior to the development of the Standards, less attention was paid to the issue of exporting hazardous electronic
waste to developing countries; this practice has been documented to have far-reaching global implications in terms
of environmental injustices and adverse health and environmental impacts. Also, post-Audit Observation interviews
indicated that  e-scrap generators as well as electronics recycling facilities generally want to comply with legal
requirements when exporting used electronics.
19 See 49 CFR Parts 100-185 and the Federal Motor Carrier Safety Regulations (49 CFR Parts 382, 383, 387, 390-397, and
40). See more at: www.fnicsa.dot.gov/regulations/hazardous-materials/how-complv-federal-hazardous-niaterials-regulations.
                                                  30

-------
Opportunities for Improvement. In the majority of Audit Observations in which export was in scope, it appeared
that export practices were not thoroughly audited. The observations identified opportunities for improvement for
auditors in reviewing the following areas during the audit:

    •  Export records throughout the recycling chain rather than focusing on the Recycler's first-tier immediate
       downstream vendor(s);
    •  The validity of the consent (including expiration dates) of the foreign Competent Authority to accept the
       proposed or actual export;
    •  Downstream audit packets/reports to verify conformity to export requirements (auditors were observed
       simply accepting the presence of an audit report to indicate conformity); and
    •  Issuances of non-conformity if the company exporting CRTs for recycling is not on EPA's approved list
       (available on EPA's website).

For electronics recy cling facilities, it appears there is an opportunity to improve:

    •  Record-keeping regarding exported shipments;
    •  Obtaining and retaining Competent Authority and Acknowledgement of Consent notices (export, transit,
       and import) in a timely and updated fashion; and
    •  Reviewing the EPA website for notifications  regarding companies approved to export CRTs for reuse or
       recycling.

The Study found that there appears to be a lack of clarity in the industry about the implementation of the U.S. export
laws and OECD  Decision C(2001)107/FINAL (referenced by both R2 and e-Stewards), and the Basel Convention
and the Basel BAN Amendment (referenced by e-Stewards), and challenges understanding these compliance and
operational requirements.   Post-Audit Observation interviewees noted a need for better tools, guidance and
information to be able to better understand and thus comply with these requirements in the Standards.

Key Takeaways.  Audit observations and interviews both indicated that electronics recyclers' management systems
and auditing of export requirements by CB auditors are not wholly achieving the intent of either Standard. There is
a need to increase training  and awareness of export requirements  (particularly for CRTs), and to  enhance the
operational knowledge of certified electronics recyclers and  auditors, in relation to import-export and legal
compliance know-how.
 (2.11) EXPORT RESTRICTIONS FOR FOCUS MATERIALS AND HAZARDOUS ELECTRONIC WASTE
 ISSUE
SUGGESTED STRATEGIES FOR IMPROVEMENT
 Implementation,
 training and
 awareness
   Create tools  such as a  lockbox  or other confidential  means by which auditors can verify
   information without disclosing commercial terms or confidential business information (CBI).
   Develop and conduct additional training for auditors on how to evaluate compliance with export
   requirements, including using case studies.
                                                  31

-------
2.12.  Site Closure and Insurance
Description. Each of the Standards includes controls for site closure and financial assurance to ensure the proper
closure of facilities and to prevent abandonment of electronics. Adequate insurance is required to cover processing
risks, health and safety risks, environmental liability, professional liability and data liability, as appropriate. While
it might seem like a 'Plan' step, keeping a closure plan up to date makes this part of the 'Do' phase of the Plan-Do-
Check-Act system.

Strengths.  Good practices were observed, including instances in which auditors reviewed certificates of insurance
and checked the expiration dates. In addition, some auditors spent considerable time exploring indemnification and
how it was communicated to customers.

Opportunities for Improvement. Generally conforming  practices were observed. However, opportunities  for
improvement at facilities exist in the following areas:

    •  Closure for an additional warehouse was not included in the Closure Plan at one facility;
    •  Several facilities  did not  address environmental sampling and remediation in their Closure Plans, which is
       required by both standards.

For auditors, it appears there is an opportunity to improve:

    •  Verification/validation of the value of an electronics recycler' s facility assets used as a closure mechanism
       (i.e., the auditor should verify that the value of equipment in the facility that would be sold in the event of
       a closure is as stated; ideally this should be independently verified);
    •  In one case, an auditor did not question details of a closure plan that appeared (to the Audit Observer) to
       be inadequate; and
    •  In  addition, some assets listed, such as vehicles or  movable equipment,  could be removed before
       abandonment, and thus should not be counted as a closure asset.

It is important to note that because Closure Plans are not legally binding (unless there is a state-required program),
in a bankruptcy situation, sale of assets may  go to banks that have liens or to other creditors. It was not clear, based
on observations, that recycling facilities or auditors understand this when reviewing a facility's closure plan that
states that the value of assets would offset closure. This is an area for further improvement within the entire system.

Key Takeaways. This area would benefit from increased scrutiny by all stakeholders, in order to create knowledge,
enhanced site closure assurance practices, and more thorough auditing of this element by CB auditors. Case studies
or recommendations regarding appropriate  insurance limits and instruments focusing on site  closure "financial
surety" would be helpful.
(2.1 2) SITE CLOSURE AND INSURANCE
ISSUE
Strength of closure plans
and financial assurance
requirements; better
understanding of this by
stakeholders
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Develop case studies or recommendations regarding appropriate
instruments focusing on site closure "financial surety."
• Improve auditors' and facilities' understanding of the limits of use
mechanism.
insurance limits and
of assets as closure
                                                   32

-------
2.13.  Downstream Accountability
Description. Specific provisions in each Standard require electronics recyclers to conduct initial and ongoing due
diligence of and accountability for the practices of all downstream recyclers, particularly for  FMs, HEWs, and
PCMs. Thorough record-keeping  is critical for this subject area.  Differentiated from Section  2.10, Materials
Recovery  and Disposition (auditors look at how materials are  processed and disposed per  the  Standards'
requirements), in the Downstream Accountability section auditors look for how facilities select, audit, and document
their downstream vendors' practices, to be assured the Standards' requirements are met.  This is a very important
part of the 'Do' phase of the Plan-Do-Check-Act system, and is unique to the electronics recycling Standards.

Strengths. A number of systems strengths were observed during the audit observations, including instances in which
auditors:

    •  Reviewed auditor qualifications for those conducting downstream audits;
    •  Conducted a thorough exploration of all downstream material flows and verification of conformity at each
       stage of the recycling chain;
    •  Verified signed agreements between the recycler and downstream vendor;
    •  Issued non-conformities due to issues such as incomplete due  diligence records, downstream processors
       not agreeing to transparency as required; and lack of contracts with downstream vendors; and
    •  Verified contracts with downstream vendors conducting reuse tasks.

In addition, the Study identified an emerging good practice in whichfacilities are setting up software programs that
will prohibit shipping to destinations that are not approved in the software system.

Opportunities for  Improvement. Observations  indicated that  verification  of downstream  accountability
requirements varied, with opportunities for improvement for auditors in the following areas:
    •  Confirming that all downstream material flows are identified throughout the recycling  chain, beyond the
       first downstream vendor and especially for vendors conducting reuse activities;
    •  Interviewing local facility representatives involved in contracting and shipping to downstream to verify
       implementation of corporate downstream accountability processes;
    •  Reviewing downstream audit packets/reports to verify conformity of requirements rather than simply
       accepting the presence of an audit report to indicate conformity;
    •  In one instance, an auditor  failed to review shipping records at all and also simply noted verbal reports from
       the facility about downstream accountability, without checking records;
    •  Improving the sampling of shipping records by:

           o  Ensuring an adequate sample size  of downstream records to verify system conformity.  This is
              thought to be a challenge related to inadequate audit time; and
           o  Conducting random sampling of downstream vendors to verify conformity, rather than relying on
              samples chosen by the recycler; and Issuing a non-conformance (NC) rather than  an area of concern
              (AOC), which may have been  a more appropriate  finding in an instance in which approval  of
              downstream vendors is not  systematically demonstrated in the management system.

Opportunities for improvement for facilities include strengthening systems of controls and better training within
facilities, to ensure that requirements are translated into procedures and practices.
                                                  33

-------
Key Takeaways. Verification of downstream accountability requirements is perhaps the most challenging element
of the Standards due to:

    •   The desire of downstream vendors to maintain the confidential business information and/or the intellectual
        property associated with their processes. When responding to an upstream certification recycler's query, a
        downstream vendor may be reluctant to be transparent (i.e., disclose details of its subsequent downstream
        vendors, its EH&S practices, its regulatory status,  its  processing technologies, etc.) for fear of this
        information 'getting out' - thus costing its customers or causing a disadvantage in relation to competition;
    •   Costs associated with adequate auditing of downstream vendors;
    •   The size and/or fluidity of some facilities' downstream recycling chains; and
    •   The need for auditors to randomly sample sufficient  numbers of records to verify conformity in limited
        amounts of time.

However, interviewees concurred that attention brought to downstream due diligence  by the  Standards has
dramatically improved the electronics recycling industry.
(2.13) DOWNSTREAM ACCOUNTABILITY
ISSUE
Downstream
accountability
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Standardized audit packet guidance tool - what an upstream can ask for from its downstream,
as a basic practice in the electronics recycling industry.
• Develop easier-to-use and standardized tools or requirements for mass balance reports (e.g.,
new e-Stewards standard requires linking shipping records to tracking information).
• Tools to keep transparency of downstream vendors but which respect commercial privacy.
• Consider audit queries by auditors for downstream sampling to go beyond getting data from
electronics recyclers' EH&S staff (e.g., ask Accounting).
2.14. Monitoring and Measurement
Description. This important area ensures organizations are monitoring and measuring, on a regular basis, their
activities that have environmental and health and safety (EH&S) impacts. Monitoring also includes calibration of
equipment, industrial hygiene testing, tracking EH&S  performance, tracking of equipment, logging progress on
objectives and targets, and periodic management reviews. While 'measurement' sounds like 'checking' it is a part
of the 'Do' phase of the Plan-Do-Check-Act system.

Strengths.  Many good practices were  observed including the use of very thorough monitoring checklists by
facilities for (i) workplace checklists  (e.g., such as weekly, monthly, quarterly and/or annual EH&S inspections),
and (ii) daily hygiene checklists. In addition, strengths  were observed by auditors who reviewed the management
review process in which objectives and targets are monitored and checked the status of scale calibrations.

Opportunities for Improvement.  Generally conforming practices were observed.  However, the observations
indicated that opportunities for improvement for facilities exist in the following areas:

    •   Identifying testing equipment in testing and refurbishment areas in need of calibration;
    •   More frequent health and safety monitoring. Several non-conformances were issued including blocked
        electrical panels and two facilities where fire  extinguishers missed monthly inspections; and
    •   Auditors noted that while companies have thorough EH&S checklists, they often do not follow-up and take
        care of issues identified via the checklists.
                                                  34

-------
Opportunities for improvement for auditors were observed as follows:

    •  Requesting monitoring records. During one audit, monitoring records were not requested by the auditor.

Key Takeaways. Observations and interviews both indicated that monitoring and measurement is understood and
implemented by facilities and audited effectively by auditors.
(2.14) MONITORING AND MEASUREMENT
ISSUE
Effective use of
monitoring and
measurement tools
and systems
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Ensure compliance with legal requirements is transposed into periodic monitoring
which are used regularly.
• Standardized monitoring and measurement software for the electronics recycling
operational and compliance areas.
/ measurement tools
industry, crossing all
2.15. Tracking
Description.  Tracking is part of the 'Check' stage in the Plan-Do-Check-Act cycle. Each of the Standards requires
facilities to have 'controls' (or well-documented systems) for tracking electronics throughout the recycling chain
and to reuse  markets, which the facility should be checking periodically to ensure conformity to the Standard.
Tracking helps ensure material is going to acceptable markets and to qualified downstream vendors, and CB auditors
look for these controls to verify conformity.  Tracking differs from downstream due diligence as it is part of the
'Do' phase (Section 2.13) instead of the 'Check' phase.

Strengths. Good practices observed included:

    •   Facilities using bar-coding for inventory management and/or other custom software to ease work related to
        tracking of material flow;
    •   Auditors sampling the mass balance of specific materials such CRTs and batteries; and
    •   An auditor noted that e-Stewards' mass balance  calculations and the requirement for transparency have
        transformed the industry.

Opportunities for Improvement. Generally conforming practices were observed, however, observations indicated
that opportunities for improvement for facilities exist in the following areas:

    •   Tracking of reusable electronics (it can be difficult to reconcile pounds in with units out or units in and
        pounds out);
    •   Providing auditors with better evidence that customers may obtain downstream flow chart if requested; and
    •   Assembling better evidence of monitoring and control of the destinations of FMs/HEWs.

It appears that  there may be a significant opportunity for improvement in verification by  auditors of tracking
material through to final disposition, for example:

    •   In several cases it appeared that auditors did not review the mass balance of equipment going for reuse;
    •   Inquiring if the electronics  recyclers' customers requested  and were provided  downstream vendor
        information; and
    •   Auditors mentioned they did not have adequate time to conduct the mass balance in debriefs after the Audit
        Observations.
                                                   35

-------
Key Takeaways. Observations and interviews both indicated that tracking is perhaps one of the most challenging
aspects of the Standards. Without doing a thorough accounting audit of all transactions into facilities and all
transactions out of facilities, within the given mass balance time period, there is room for misrepresenting data in a
category that does not get sampled (e.g., where an auditor might miss the fact that a facility is illegally stockpiling
CRTs by not tracking transaction documentation thoroughly).
 (2.15) TRACKING
 ISSUE
SUGGESTED STRATEGIES FOR IMPROVEMENT
 Tracking to
 downstream
 destinations -
 adequate time to
 audit
    Ensure auditors have control of sample selection when tracking to downstream destinations
    (and have adequate time to audit).
    Fine-tune formulas for audit time to ensure adequate time for sufficient sample size review and
    analysis. Formula could consider the number of downstream vendors, volume, complexity of
    material, complexity of organization.  Standards owners could do an analysis of adequacy of
    audit time - in relation to this specific issue.
2.16.  Evaluation of Compliance
Description. Certified facilities are required to have a system whereby they evaluate their own compliance with
legal requirements including environmental health and safety, data security, and export requirements. This area is
also part of the "Check" stage in the Plan-Do-Check-Act cycle. Specifically, facilities are required to have a
procedure that spells out how recyclers periodically check that they are in compliance with legal and other
requirements. This ensures that facilities do not just identify their compliance requirements, but also assess whether
their operations and processes are in compliance. An auditor will look for items like a list of legal compliance
requirements; an independent audit assessing conformity with legal requirements; certificates, licenses, and reports
proving up-to-date conformity; and management/employee knowledge of applicable regulations.

Strengths. This provision is causing certified electronics recyclers to better know and understand complex legal
requirements that apply to their facilities.

Opportunities for Improvement. Opportunities for improvement by auditors include ensuring that this element is
audited and asking to see the procedure for evaluating compliance with export, legal and other requirements. For
the facilities, several additional opportunities were identified and indicated the need to:
    •    Demonstrate compliance with this requirement.   One facility had  not had an external audit of its legal
        compliance  conducted since 2011, had not conducted any internal reviews since that time,  and its list of
        legal requirements was not up-to-date; and
    •    Understand  which laws apply and which do not.  For example, one facility missed having  stormwater
        compliance  evaluation on its legal requirements, but had  items listed that did not apply to the facility (a
        Treatment Storage and Disposal Facility closure plan).

Key Takeaways.  As mentioned under System Planning (Section 2.2), clear identification of legal and other
requirements and environmental health and safety risks is fundamental for an effective management system. This
requirement ensures that facilities do not just identify their compliance requirements but assess if they are in
compliance. For facilities new to having an ISO 14001-based EH&S Management System, the "Check" part of the
cycle  can be a challenge.  The 'evaluation of compliance' requirement helps ensure  some 'checking' goes on.
Suggested strategies for improvement for improvement center on training and awareness. It also was noted  by
interviewees that better enforcement of existing laws by regulators would motivate facilities to use this function to
their advantage.
                                                   36

-------
(2.16) EVALUATION OF COMPLIANCE
ISSUE
Understanding how
evaluation of
compliance works
and putting it into
better practice
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Better enforcement of existing laws by regulators would encourage
compliance more proactively and thoroughly.
• See suggested strategies for improvement under (2) System Planning
on 'how to do this.'
• Ensure "evaluation of compliance" procedures include looking at export
security - these areas are often overlooked.
facilities to "check" their
but also provide training
, transportation, and data
2.17. Nonconformity,  Corrective, and Preventive Action
Description. Also part of the 'Check' stage in the Plan-Do-Check-Act cycle, facilities are required to have and use
tools to assess internal nonconformity with their EHSMS, and to take and document corrective and preventive
actions. This system helps ensure facilities proactively put themselves back on track when they stray - which in
turn reinforces organizational knowledge of what conformity means -leading to effective change and improvement
in the EHSMS.

Strengths. Stakeholders in the system  appear to generally understand this requirement and implement it well.
Facilities were observed that responded to their non-conformities in a timely and focused way, using their corrective
action logs and their continual improvement and preventive action logs effectively. One facility uses its external
consultant to verify Corrective and Preventive Actions (CAPA) have been closed from all items  on the facility's
non-conformance log.

Opportunities for Improvement. This area was generally observed to be well understood by both facilities and CB
auditors. Auditors noted when facilities' non-conformances (NCs) from previous audits had not been correctly
transposed into CAPA reports and also noted when NCs were closed in a timely and effective fashion. Our findings
and those of the astute auditors noted the following areas of improvement for facilities, including:
    •    Correctly using the root cause analysis in a facility's CAPA reports. Failing to do this  means a facility
        may not identify more important underlying areas needing improvement;
    •    Generating CAPAs related to actual operations versus  only from the CB audits;
    •    Ensuring documented changes to related Procedures and Training Materials are implemented;  and
    •    Reviewing CAPAs at management meetings.

Key Takeaways. Although the use of nonconformity tracking and CAPA reports is one  of the  best-established
elements in the entire system, there is room for tools to be developed to further enhance this element. Such tools
might include training in root cause analysis specific to  the electronics recycling industry, templates for internal
nonconformity assessments, or other similar mechanisms.
(2.17) NONCONFORMITY, CORRECTIVE AND PREVENTIVE ACTION
ISSUE
Correct use of and
auditing of
nonconformity,
corrective and
preventive action
tools
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Develop easier-to-use tools for doing CAPA-related work - e.g., apps for smart phones and
tablets, software for facilities (noting that operational folks are less inclined to use paper and
forms).
• Periodic review of nonconformity, corrective and preventive action requirements and best
practices for auditors as well as facilities, perhaps as part of CEU-style requirements.
                                                 37

-------
2.18. Control of Records
Description. As part of the 'Check' stage in the Plan-Do-Check-Act cycle, facilities are required to implement tools
to track and control all records that demonstrate conformity to the Standards to which they are certified. Records
demonstrate evidence of completing a required task such as training records, inspection records or management
review records. A procedure must be established for the identification, storage, protection, retrieval, retention and
disposal of records. Records must also be used to demonstrate the flow of materials through the facility.

Strengths. Facilities were observed that properly completed their materials flow records at each month-end, along
with a physical inventory.

Opportunities for Improvement. Opportunities for improvement by auditors in this area were noted as follows:

    •  Ensuring that this requirement is not overlooked (in one case this requirement was not audited);
    •  Sampling an adequate number of records to be assured the facility's system is working. This limitation in
       sampling is attributable to time limitations for the audit; and
    •  Checking actual documents. One auditor simply relied on what the facility stated was in its records, rather
       than viewing them directly.

Opportunities for improvement for the facilities included attending to this detail-oriented task and using it to ensure
alignment between procedures, actions, and Records Control documents.

Key Takeaways. This is a basic element of running a well-managed EHSMS. Failure to do this indicates a lack of
attention to detail that might indicate problems elsewhere in a facility with more serious consequences.
 (2.18) CONTROL OF RECORDS
 ISSUE
SUGGESTED STRATEGY FOR IMPROVEMENT
 Correct control of
 records
    Develop easier-to-use tools for records control - e.g., apps for smart phones and tablets. Use
    scheduling tools as part of improving operations.
2.19. Internal Audit
Description. The internal audit, conducted by trained staff or a second-party auditor, should provide useful
information on whether the facility's EH&S management system is well-implemented, reviewed adequately, and
conforms to the requirements of the Standards to which the facility is certified.  Internal audits of an electronics
recycling facility, regardless  of who conducts them,  should provide management with identification of non-
conformities and information on deficiencies, along with recommendations for system improvement. Internal audits
are a critical part of the 'Check' phase of the Plan-Do-Check-Act system,  enabling management to monitor key
activities, identify and correct problems, and gauge ongoing progress.

Strengths. The internal audit requirement has led to facilities having more thorough review of their operations than
just the annual audit by a Certifying Body. This, in turn, has resulted in more opportunities for facilities to improve
operations and practices within their environmental health and safety management systems.
                                                  38

-------
 Opportunities for Improvement. Opportunities for improvement for auditors were noted as follows:

    •   Verifying conformity between the internal audit plan and the internal audit report;
    •   Following through on discrepancies between observations in the internal auditor's report versus  actual
        practices in relation to export and downstream due diligence;
    •   Completing queries to facility about how it follows up on its internal audit findings (i.e., determining if
        facility acted on NCs, AOCs, or Opportunities for Improvement from internal audit); and
    •   Adequately determining competency and credentials of the internal auditors.

 Key Takeaways.  If auditors focused more on ensuring internal audits are carried out in conformance with the
 Standards' requirements, facilities will be more likely to take the findings of the internal audits seriously, and use
 those findings to implement improvements. Building capacity for internal audits within organizations would support
 facilities using the internal audit as a valued management improvement tool. Better clarification of minimum
 competency requirements for internal auditors from the Standards owners would help.
(2.1 9) INTERNAL AUDITS
ISSUE
Internal audit
issues
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Use CEUs as tool to ensure continued training in key topic areas, such as how
conduct an internal audit, qualifications of internal auditors, how to effectively use
of an internal audit.
to effective
the findings
• Offer more frequent training (including training for new areas highlighted above), which should
lead to better availability of skilled internal auditors.
• Clarification of minimum competency requirements for conducting internal audits.
2.20.  Management Review
 Description.  As the 'Act' stage in the Plan-Do-Check-Act cycle, Management Review requires management to
 provide periodic reviews of the facility's entire environmental health and safety management system, and to keep
 records of those reviews. The Management Review includes the assessment and implementation of any changes to
 the EHSMS deemed necessary by the Internal Audit.

 Strengths. Facilities were observed that have moved from annual to more frequent management reviews.  One
 facility shifted  to  monthly  management review  meetings (with corresponding better  documentation  and
 communications) and  saw improvements in overall EH&S management system implementation. Another facility
 effectively used management review action logs to track progress.

 Opportunities for Improvement.  Auditors  were observed  auditing this element comprehensively.  However,
 opportunities for improvement by auditors include:

    •   Meeting with facility management and inquiring about participation in management reviews; and
    •   Ensuring that management review meetings: (i) cover all required topics  (e.g., compliance evaluation,
        follow-up on customer complaints, completions) by reviewing minutes; and (ii) include required outcomes
        (i.e., Action Plans related to deficiencies discussed).

 For facilities, opportunities for improvement were also observed, as follows:

    •   Demonstrating  a serious commitment  to  use management  reviews  to  actually ensure the EH&S
        management system is being implemented correctly by all staff;
                                                  39

-------
    •  Addressing all outstanding NCs, OFIs, and AOCs which have been documented during both external and
       internal audits with full follow-up and accurate documentation; and
    •  Using management review to follow-up on serious matters (e.g., OSHA recordable incidents, regulatory
       compliance matters).

Key Takeaways. Establishing and sustaining complete management  support for environmental health and safety
management systems in the electronics recycling industry has proven challenging. Interviews and audits suggested
that causes for this include managers' and/or owners':
    •  Limited understanding of risk management and the value of accurate risk assessment.
    •  Difficulty with effectively implementing a successful EH&S management system, which can be a change
       not just in operations but in company culture.

Suggested actions for addressing these causes include a focus on training and awareness for facility managers and
owners.
(2.20) MANAGEMENT REVIEW
ISSUE
Management
commitment and
company culture
SUGGESTED STRATEGIES FOR IMPROVEMENT
• Target electronics recycling facilities' management via webinars, conferences, and other
professional settings, offering training specific to management and owners - not just EH&S
managers.
• Research how managers can change company culture and employee behavior when
implementing an EH&S management system, specifically for the electronics recycling industry;
disseminate findings through trainings.
2.21. Additional  Findinas from Stakeholder Interviews
This section documents additional findings from the stakeholder interviews that were not already captured in the
discussion in Sections 2.1 -2.20. This section includes additional information on ANAB' s oversight role in reviewing
and ensuring the CBs' competence; and how the CBs select and train their auditors, information about their training
programs, and their quality assurance/quality control processes and procedures.

ANAB's Oversight Role
ANAB informs the Standards owners of changes to accreditation rules, and communication during the recent
revisions to the Standards was noted to be clear and effective. ANAB is currently the only approved Accreditation
Body to accredit CBs to certify facilities to the e-Stewards and R2 standards. ANAB has worked with R2 since
2008 and e-Stewards  since 2009.  A Certification Body's management of competency is governed by ISO
17021:2011: Conformity Assessment Requirements for Bodies Providing Audit and Certification of Management
Systems. ISO  17021 was developed in part by ANAB, and is used by ANAB to ensure the CBs meet the technical
requirements, and competency criteria to audit to e-Stewards and R2.

ANAB's oversight process includes the periodic review of the CBs' processes to ensure they are conforming to ISO
17021  and  the requirements of  each Standard.  This  process includes  assessors, administrative roles, and
accreditation council and subcommittees that are involved in the management review and accreditation process for
the CBs.  ANAB assessors are required to have similar competencies as the CB auditors.  Once all the information
is gathered (appropriate documentation from the CBs' management operations and the results of witness audit(s)),
ANAB program and  accreditation management make the determination that the  CB  is conforming  to  the
requirements of ISO 17021. Interviews also indicated that ANAB has an effective process in place to ensure regular
communication with both R2 and e-Stewards.
                                                 40

-------
Certifying Bodies
Interviews with the six CBs revealed that all CBs have training programs in place to ensure their auditors are well
trained in the requirements of auditing an environmental management system (ISO 14001) as well as the electronics
recycling-specific requirements of both of the Standards. The CBs utilize the effective core training programs and
resources offered by the Standards owners or third-party training organizations, with many providing additional in-
house training to further enhance auditor knowledge and competency. Interview findings also indicated that all six
CBs have solid quality control and quality assurance programs and procedures in place to verify their auditors'
competency prior to conducting audits in the field, and also to ensure proper review of auditors'  reports that
document facility audit findings, major and minor non-conformances, and recommendations for certification.
3.  Data Analysis
Information from the interviews and audits was aggregated to identify patterns of strengths and opportunities for
improvement. The  Checklist data from the audits contained a combination of ratings from 1 to 5. The data were
aggregated across all nine audits for each of the 144 discrete elements audited (See section 9 of Appendix B for a
listing of all discrete elements). The data were then aggregated by each of the 20 topic areas to determine an average
overall score between 1 and 5. The  discrete elements in each topic area were weighted according to the number of
audits addressing that element, to account for the fact that not every topic area was audited in every audit due to the
different audit types as well as the different versions of the Standards. EPA carefully reviewed the audit findings -
as well as the findings from the interviews - to identify patterns of strength, opportunities for improvement, and
other trends.

Table 3.1 details the aggregate scores across all audits in each of the topic areas. These scores were determined by
calculating the average score across all discrete elements in a given topic area, with each discrete element weighted
according to the number of audits in which it was assessed. For example, if Topic 1, Element 1 received an average
score of 4.75 across nine audits, and Topic 1, Element 2 received an average score of 3.25 across six audits, then
Topic 1 would receive an aggregate score of 4.15 using the formula:

                                            aฑx + a2y + •••
                                              % + a2 H—

In this formula, "a/' is the number of audits where Element 1 was assessed, "02" is the number of audits where
Element 2 was assessed, "jc" is the average score for Element 1, and "j" is the average score for Element 2.

Table 3.1: Aggregated Audit Checklist Scores by Topic Area
Topic Area
1. System requirements
2. System planning
3. Training and communication
4. Operational control
5. Health and safety
6. Reuse
7. Data security and destruction
8. Management of FMs, HEWs, and PCMs
9. Emergency preparedness and response
10. Materials recovery and disposition
11. Export restrictions for FMs and HEWs
12. Site closure and insurance
13. Downstream accountability
14. Monitoring and measurement
15. Tracking
# Discrete
Elements
2
16
3
7
23
15
11
9
3
9
2
12
9
4
7
Weighted Average
Audit Score (1-5)
5.00
4.18
4.32
3.77
3.87
4.22
4.21
4.04
4.30
4.06
3.00
4.44
3.85
4.54
4.12
                                                  41

-------
Topic Area
16. Evaluation of compliance
17. Nonconformity, corrective and preventive action
18. Control of records
19. Internal audit
20. Management review
# Discrete
Elements
2
3
3
3
1
Weighted Average
Audit Score (1-5)
3.38
4.83
4.22
4.28
4.33
 3.1.   Key System Strengths
The data aggregation and analysis from the interviews and audits identified a number of strengths in how the
system functions as a whole:

•   Updates and improvements over time.  Both Standards are committed to continual improvement. During the
    past few years, as they have grown and become more established, both Standards owners have listened to
    stakeholders, conducted research,  published updated  and improved  versions,  begun observing audits  in a
    witness capacity, and implemented programs providing tools and resources to their certified recyclers. In
    addition, both Standards have updated their organizational and operating structures, and continue to do so, to
    better serve their constituents. The Standards are engaged in the training regimes for CB auditors, and are
    continually expanding training offerings for electronics recycling facility operators.

•   Clear understanding of roles and responsibilities.  Stakeholders in the electronics recycling certification
    system know their roles and carry them out with integrity. Certification bodies appear to work well with ANAB
    and work to keep up with issues in the industry. CB management appears to understand the challenges faced
    by their auditors and the structural limitations in the system (such as auditor time constraints). This Study
    observed many hard-working, diligent, focused auditors using their limited time well.  Auditors were observed
    generally  tracking  audit  trails  well;  performing  sound  auditing  of: records  control; monitoring  and
    measurement; corrective and preventive action procedures; and facilities' tracking systems for incoming and
    outgoing materials flows.

•   Receptive to feedback. Certified  electronics  recycling  facilities  are  particularly  strong at giving clear,
    thoughtful, and honest feedback, including constructive comments to other system stakeholders. Further, many
    facilities are patiently taking both management and staff up the steep but necessary learning curves needed for
    certification for difficult requirements such as legal and environmental  compliance,  occupational health and
    safety mandates and best practices, export matters, and downstream due diligence practices. Managers are
    demonstrating ongoing willingness to learn how to effectively run a Plan-Do-Check-Act management system
    in an industry new to systematic management systems.

•   ANAB's role is clear and effective. It appears that the ANAB system of accrediting CBs is generally working
    well. The ANAB witness auditors  and the  CB auditors permitted the EPA to sit in  on the de-briefs ANAB
    provided immediately after the CB auditor finished the audit. In both instances,  the ANAB witness auditor
    demonstrated thorough knowledge of the Standard  being audited, provided constructive  feedback,  and
    identified useful areas of non-conformance,  and opportunities for improvement.

With regard to the observed audits, Table 3.2 below provides highlights specific of areas of strength in both
Standards that were observed to be audited comprehensively and consistently across multiple audits. Note that
Section 2 of this report provides a detailed assessment of the strengths observed within  each topic area, and Table
3.2 is not intended to be an all-inclusive list.
                                                   42

-------
Table 3.2: Key Areas of Strength
Topic
Area*
1.
7.
14.
17.
Topic Area Name
System
requirements
Data Security/data
destruction
Monitoring and
measurement
Nonconformity,
corrective and
preventive action
Discussion
• Facilities are devising Environmental, Health, and Safety
Management Systems (EHSMS) that cover the entire
scope of the applicable Standard(s), and are ensuring that
EHSMS are kept up-to-date.
• The updated versions of the Standards have improved the
detail and requirements for this area, and very thorough
auditing of this topic area was observed.
• Auditors are ensuring facilities are monitoring and
measuring, on a regular basis, their activities that have
environmental and health and safety impacts.
• Auditors are ensuring the facilities assess internal
nonconformity with their EHSMS, and are taking corrective
and preventive action.
Weighted
average
audit score
(1-5)
5.00
4.21
4.54
4.83
3.2.  Key Opportunities for Improvement
While  the Standards generally  are being implemented well,  the  analysis  of patterns and trends identified
opportunities for improvement in a few key topic areas, which are outlined below in Table 3.3. These areas were
all discussed in detail above in Section 2, and are linked to specific suggested strategies for improvement in that
section as well as summarized in Appendix A. As previously mentioned, it is important to note that the auditing
process is highly qualitative, and the average overall score for each topic area reflects EPA's assessment of how
well that particular topic area of the Standards was audited based only on the nine observed audits. The score is an
indicator  - but not the only indicator - of the overall effectiveness/strength of that topic area. The qualitative
information provided through the structured stakeholder interviews also was carefully reviewed to determine trends,
such as frequently mentioned strengths and opportunities for improvement, in the implementation of the Standards.
Thus, EPA considered both the combination of the audit findings and the interview responses in identifying patterns
of strengths, opportunities for improvement, and other trends.
Table 3.3: Key Opportunities for Improvement
  Topic
  Area*
    Topic Area
                    Discussion
Weighted
 Average
  Audit
  Score
    3.
Training and
communication*
Auditor and facility training, understanding and consistent
implementation of:
 •   Environmental health and safety hazard risk reduction
     strategies;
 •   Hazardous waste requirements;
 •   Import/export requirements (especially for Cathode Ray
     Tubes (CRTs); Organization for Economic Cooperation
     and Development (OECD) countries vs. non-OECD
     countries);
 •   Federal, state, and local legal requirements (e.g.,
     Universal Waste rule); and
 •   Proper materials management (including batteries).
   4.32
                                                 43

-------
 Topic
Area*
    Topic Area
                        Discussion
Weighted
 Average
  Audit
  Score
   4.
Operational
controls
    Auditor verification of implementation of written procedures
    such as how to manage FMs and how to best mitigate
    environmental, health, and safety risks -during audits.
   3.77
   5.
Health and safety
    Auditor and facility understanding and dissemination of
    actionable information regarding human health risks posed by
    dismantling end-of-life electronics, including potential
    exposures and subsequent use of risk-based controls (e.g.,
    personal protective equipment (PPE)).
   3.87
   8.
Management of
FMs, HEW, and
PCMs
    Auditor and facility understanding of what is and is not
    hazardous or universal waste under state and federal
    regulations, and the associated management requirements.
   4.04
  10.
Materials recovery
and disposition
    Facility and auditor attention to safe packaging of hazardous
    equipment (e.g., CRTs, batteries).
    Auditor verification of facilities' controls for ensuring
    downstream processors conform to Standards' requirements.
   4.06
  11.
Export restrictions
for Focus Materials
(R2) and
Hazardous
Electronic Waste
(e-Stewards)	
                                  Auditor and facility training, understanding, and awareness of
                                  export requirements (particularly for CRTs).
                                                               3.00
  12.
Site closure and
insurance**
    Auditor understanding of the form and accessibility of
    insurance (or other forms of financial assurance) in the event
    of abandonment or bankruptcy.
   4.44
  13.
Downstream
accountability
    Auditor verification of requirements in areas such as:
    confirming that all downstream material flows are identified
    throughout the recycling chain beyond the first downstream
    vendor; and improving downstream verification practices by
    auditors during an audit.
   3.85
  15.
Tracking
    Auditor performance of the mass balance calculation
    (account for all transactions into facilities and all transactions
    out of facilities) to verify equipment going for reuse.
   4.12
  16.
Evaluation of
compliance
•   Auditor review of legal and environmental, health and safety
    requirements, and compliance with those requirements.
   3.38
* Auditors assess whether or not facilities have determined their training needs and communicate them
appropriately. While these clauses were observed to be well audited, the need for further training and knowledge
development was noted across numerous topic areas and emerged as a top area for improvement from the
interviews.

** Auditors assess whether the closure plans are in place and contain the required components; however, while
these requirements were generally audited well (meaning the auditors identified the closure plans with required
elements), an important opportunity for improvement emerged to improve auditor understanding of the adequacy
of the forms of insurance (or other forms of financial assurance).	
                                                44

-------
4.  Suggested Strategies for Improvement & Top Recommendations
The Study's findings are based on the stakeholder interviews EPA
conducted and  the  audits EPA observed, which reflect  a small
percentage of all certified electronics  recycling facilities and of all
auditors at  work in the electronics recycling industry in the U.S.
Though limited in scope,  patterns  and trends nonetheless emerged
from the Study's findings, and specific strategies  for addressing the
observed opportunities for improvement are provided in Section 2 of
this report  (and are compiled in Appendix  A).  Analysis of these
patterns and trends also  enabled EPA  to  develop a set  of  top
recommendations  to  address the root  causes  of  many  of  the
opportunities for improvement, which are presented in this Section of
the report.
  Top Recommendations
Provide additional training and
guidance materials to grow the
knowledge base for all
stakeholders
Provide regular updates to the
Standards to ensure they continue
to evolve
Increase audit time to allow for
more thorough audits
Explore and address perceived
conflict-of-interest issues.
Top Recommendations
In practice, many stakeholders will continue to influence and shape the development of responsible recycling in the
U.S. and therefore have been considered when drawing the Study's conclusions and making recommendations; as
such, EPA encourages all organizations that have a role to play in shaping the standards and certification process
to consider the suggested strategies for improvement and top recommendations in the Study.  This broader group
of stakeholders includes:

    •   The Standards owners (Sustainable Electronics Recycling International and the Basel Action Network)
    •   The Certifying Bodies
    •   Federal and state regulators
    •   Trade associations
    •   ANAB
    •   EPA and other federal agencies, including the Occupational Safety and Health Administration, the National
       Institute for Occupational Safety and Hazards, and the Department of Transportation
    •   Original equipment manufacturers
    •   Academic researchers
    •   Non-governmental organizations
    •   Private-sector companies

It is important to note that because of the commitment to continual improvement by stakeholders in the system,
some of the below recommendations - as well as the suggested strategies for improvement presented in Section 2
of the report - may already be in motion.

The top recommendations are:

•  Provide additional training and guidance materials to grow the knowledge base for all stakeholders. Growing
   the base of knowledge for all  stakeholders, including auditors and facilities, is important for ensuring the
    Standards are implemented properly. In total, 85% of all interviewees - including 100% of Certifying Bodies
    and 91% of auditors - indicated that more robust training programs are needed in order to strengthen knowledge
   of health and safety risks, legal requirements - particularly related to exports and management of CRTs - and
   best operational practices in many of the 20 topic areas. Combined with the observations from the audits and
   the low weighted average scores of many of the topic areas, providing additional training and guidance materials
   to grow the knowledge base for all stakeholders in the system was determined to be one of the recommendations
                                                 45

-------
    with the most far-reaching possible impacts in improving the overall implementation of the Standards. Areas
    observed that would benefit from the development of more comprehensive additional training and guidance
    materials include:
        o   Health and safety risks. This area needs significant attention from all stakeholders in the electronics
           recycling industry and beyond (from manufacturers through regulators). Also, 45% of post-audit
           interviews, including interviews  with facility  staff, CB  auditors,  and ANAB  auditors,  indicated
           specifically that electronics recyclers need training and guidance on conducting proper risk assessments
           and hazards analyses, complying with health and safety requirements, and using appropriate controls.
           Additional research is needed into the potential health hazards of both short- and long-term exposure
           to electronics during end-of-life management.
        o   Legal requirements. Clear understanding of federal, state, and local legal and other requirements -
           coupled with an understanding of environmental, health, and safety risks - provides a fundamental
           basis for an effective environmental, health, and safety management system.  Additional training and
           guidance for all system players in this area, such as a succinct summary of all federal laws pertaining
           to e-waste, could help expand knowledge and improve implementation. Several interviewees suggested
           specifically that providing additional guidance and clarification on the CRT rule would be beneficial.

        o   Site closure and insurance adequacy. The interviews and observed audits indicated that this area would
           benefit from increased scrutiny by all stakeholders, in order to create knowledge, enhance site closure
           assurance practices, and provide for more thorough auditing of this element by CB auditors. Case
           studies or recommendations regarding appropriate insurance limits and instruments focusing on site
           closure "financial surety" would be helpful.

    Section 2 of the  Study offers suggested strategies for consideration in developing the additional training
    materials, guidance, and other tools in the context of each of the relevant topic areas.

•   Provide regular updates to the Standards to ensure they continue to evolve alongside this rapidly changing
    industry.   E-Stewards and R2  are both relatively new standards and their recent updates went smoothly.
    Updating and revising the  Standards on regular, well-publicized timetables is important to address  areas that
    could benefit from clarification in a Standard or lessons learned from prior audits. Support for more systematic,
    well-publicized plans and/or timetables for subsequent updates was indicated in 69% of all interviews, including
    73% of stakeholder interviews conducted  before  the 2013 Standards  revisions and  100%  of stakeholder
    interviews conducted after the revisions. Scheduled and publicized plans to update  the Standards will allow
    stakeholders to fully contribute and participate in the continual improvement in a fair and transparent manner.

•   Increase  audit time to allow for more thorough auditing of the Standards.  Inadequate audit  time was
    mentioned in 56% of all interviews as a limiting factor, particularly for integrated audits where R2 and e-
    Stewards are being audited at the same time as other standards, such as RIOS and ISO  14001.When asked
    specifically about the adequacy of the amount of time allowed for audits, 75% of interviewees indicated a desire
    for increased audit times, as did five of the nine facilities audited. Audit time for ISO  14001-based standards is
    specified in "IAF MD 5: Duration of QMS  and EMS Audits." To better reflect the complexity of electronics
    recycling operations, there is a need to fine-tune the formulas found in the R2 Code of Practices and e-Stewards
    Appendix C to 'right-size' audit time. Many facilities can have multi-faceted downstream material flows, export
    situations, regulatory status, reuse practices, and other variables that pose challenges to the most efficient
    auditors to adequately review materials and follow audit trails in the allotted time. Moreover, many audits cover
    multiple standards simultaneously. Increasing audit time would allow auditors to more closely examine health
    and safety monitoring, measurement,  record-keeping, and communication issues in order to  give facilities
    critical feedback on their health and safety systems.
                                                   46

-------
•   Explore and address perceived conflict of interest issues to enhance overall rigorousness of the audits. Study
    participants expressed a concern that there is a perceived conflict of interest between CBs wishing to retain their
    clients (who are the recycling facilities) and the recycling facilities themselves; that is, the CB auditors may not
    be as stringent or thorough in their audits in an effort to retain a competitive business relationship. Similarly,
    auditors being observed by ANAB Witness  Auditors may be more stringent in their findings, in an effort to
    prove competency and thoroughness; this inconsistent scrutiny may pose concerns to a facility getting more
    non-conformances than it expected when ANAB is present, for example. When asked specifically about
    whether or not they feel that the perception of a conflict of interest exists, 63% of stakeholders interviewed said
    that they did.  Exploring ways in which the key players in the system - Certifying Bodies, recycling facilities,
    the Standards owners, and ANAB - could potentially address these perceptions of conflict of interest, such as
    implementing new or different funding mechanisms, could enhance the implementation of the Standards.
5.  Conclusions
Overall, the Study suggests that the accreditation, certification and implementation process for the R2 and e-
Stewards standards is improving the responsible management of used electronics in the U.S. Through the interviews
with stakeholders across the electronics recycling system and the observed audits representing a cross-section of
facility sizes, types and services offered, the Study found that the Standards have brought order, better management,
and a growing understanding of environmental, health and safety risks, regulatory requirements, and best practices
to electronics recyclers and related stakeholders across the system.

Furthermore, interviews and audits indicate that the roles and responsibilities among the key implementers in this
system - ANAB, the Standards owners, the six CBs and their auditors, and the recycling facilities - appear to be
clear and effective, and opportunities for constructive feedback are integrated throughout the system.

    •  ANAB's system of accrediting CBs appears  to be generally working well.  ANAB's oversight witness
       auditors demonstrated thorough  knowledge  of the Standard(s)  being  audited, provided constructive
       feedback, and identified useful areas of non-conformance, and opportunities for improvement.
    •  CB interview findings indicate that all six CBs appear to have consistent core requirements for selecting
       and training their auditors;  have implemented training programs covering the  critical elements of the
       Standards (supported by the tools and training programs developed by the Standards owners), and have in
       place quality assurance and quality control protocols for reviewing auditors' findings (major and minor
       non-conformances, and recommendations pertaining to whether  a facility  achieves certification to the
       Standards).
    •  Auditors were observed to be very hard working and diligent, using their limited time on the audits well.
    •  Recyclers were observed to  be operating with a willingness to learn complex new subject matter, such as
       legal and  environmental  compliance, downstream due diligence practices, export  requirements, and
       occupational health and safety practices.

The Study identified opportunities to improve the implementation of the Standards in a few key areas, including
improving stakeholders' knowledge of health and safety risks, understanding of legal requirements, and awareness
of hazardous substances in electronics.  One central theme that emerged from the Study was that the knowledge
base needs to continue to grow and tools are needed for all stakeholders (for auditors to more efficiently audit, for
facilities to more readily operationalize improvements in these areas, for regulators to better inform the regulated
community), in the following areas:

    •   Awareness and understanding of all applicable federal, state, and local legal requirements (e.g. Universal
        Waste  and  CRT rules  and associated labeling  and management  requirements;  as  well  as export,
        communications, and documentation requirements);
                                                   47

-------
    •   Health and safety risks and hazard assessments, management, and best practices (e.g. information on what
        hazardous substances are present in electronic equipment and components, and the risks and best practices
        associated with various types of processing techniques; and
    •   Knowledge of potentially hazardous processing technologies.

The Study offers suggested strategies to address the opportunities for improvement identified in the Study, which
are presented in Section 2 of the report and are compiled in Appendix A. EPA's top recommendations, which are
intended to address the root causes of many of the opportunities for improvement, are to: provide additional training
and guidance materials in key topic areas highlighted in this report; regularly update the Standards to ensure they
continue to evolve alongside this rapidly changing industry; increase audit times to allow for more thorough audits;
and explore and address perceived conflict of interest issues to enhance the overall consistency and rigorousness of
the audits.

EPA remains committed to continuing the dialogue started by this Study and supporting the continual improvement
of the implementation of the electronics recycling standards, and will provide assistance and support to stakeholders
in discussing and implementing the recommendations outlined in this report.
                                                   48

-------
Appendix A. Compilation of Suggested Strategies for Improvement by Topic Area
Below is a compilation of the suggested strategies for improvement by topic area that are presented in Section 2 of this report. It is important to note that
because of the commitment to continual improvement by stakeholders in the system, the suggested strategies for improvement may already be in motion.
ISSUE
SUGGESTED STRATEGIES FOR IMPROVEMENT
(1) SYSTEM REQUIREMENTS
Aligning Scope statement in
Certificate with company
actions
• Ensure that this high-level item is reviewed in seminars and educational settings instructing e-cyclers and auditors about
certification to ensure that relevant aspects of company operations are audited effectively.
(2) SYSTEM PLANNING
Identification of legal and other
requirements
CRT Rule
• Create a database for auditors identifying key legal requirements for each of the jurisdictions in which the CB operates. The
database could potentially generate state-specific checklists prior to an audit. Auditors need related guidance regarding
expectations for the level of detail and comprehensiveness.
• Provide a clear summary of federal laws pertaining to e-Waste processing, management, transportation, and export; conduct
webinars on legal requirements.
• Enhance evaluation of auditor competency regarding knowledge of legal and other requirements, as part of auditor training
requirements.
• Develop outreach and training materials to clearly explain the CRT rule.
(3) TRAINING AND COMMUNICATION
Training and communication
• Develop best practices or standardized, easy-to-follow training tools for electronics recycling industry stakeholders (CBs,
recycling facilities) that covers the basics of health and safety, operational controls, data security and destruction, emergency
response and preparedness, legal requirements, management of FMs, HEWs and PCMs, etc.
• Provide consistent and 'certified' (i.e., Continuing Education Units (CEU)-style) training (covering health and safety,
operational controls, data security and destruction, emergency response and preparedness, legal requirements,
management of FMs, HEWs and PCMs, etc.).
(4) EFFECTIVE USE OF OPERATIONAL CONTROLS
Operational controls
• Use training tools and best practice case studies to build better understanding for facilities of howto use operational controls.
• Use training tools to build better understanding for auditors of howto effectively audit if operational controls are being used
correctly.
                            Implementation Study of the Electronics Recycling Standards: R2 and e-Stewardsฎ
                                                      Appendix A-p. 1

-------
ISSUE
SUGGESTED STRATEGIES FOR IMPROVEMENT
(5) HEALTH AND SAFETY
Health and safety conformity
and compliance
• Improve compliance with and enforcement of occupational health and safety laws and requirements.
• Develop training and guidance for facilities on conducting comprehensive risk assessments and hazards analyses; use of
SDSs; applicable legal requirements; monitoring results; machine guarding and lockout/tag-out; etc.
• Develop ongoing mechanism to identify hazardous substances in electronic devices as they evolve.
• Provide training on transportation requirements for hazardous substances (e.g., FAA restrictions on lithium-ion batteries in
airtransport).
• Publish research-based baseline requirements for Pb and other toxic substances sampling for human health and indoor
environment, specific to the electronics recycling industry.
• Provide training and guidance for facilities and auditors on industrial health monitoring, measurement, record-keeping, and
communication.
(6) REUSE
Functionality testing
• Provide better information/tools to ensure export of equipment for reuse and refurbishment complies with the Standards.
• Develop lists with clear definitions of various functionality tests and key component testing to assist facilities and auditors.
• Provide an improved definition of acceptable test methods, more clarification as to what is acceptable, and more guidance
on different types of testing (e.g., scenarios, flowcharts, or guidelines of good practices).
(7) DATA SECURITY/DATA DESTRUCTION
Protection of data from theft or
loss
• Develop training on risks of data theft or loss during transportation, especially with outsourced logistics and drivers-for-hire.
• Develop guidance and training on risks associated with data devices being stored onsite, including best practices for facility-
level security.
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewardsฎ
                             Appendix A- p.2

-------
ISSUE
SUGGESTED STRATEGIES FOR IMPROVEMENT
(8) MANAGEMENT OF Focus MATERIALS, HAZARDOUS ELECTRONIC WASTES, AND PROBLEMATIC COMPONENTS AND MATERIALS
Awareness of appropriate
management practices for
hazardous components
• Encourage all electronic equipment manufacturers to disclose hazardous constituents at the bill-of-materials level to
electronics recyclers, to best protect human health and environment.
• Conduct research into and create incentives for solutions for safe removal of hazardous components. Could include
developing training/webinars, including best practice case study materials for each type of hazardous material and
component, to support appropriate management of hazardous constituents.
• Better attention to safe dismantling of liquid crystal displays (LCDs). The potential for breaking the small, fragile, mercury-
bearing tubes is high.
• Conduct or support research into protocols for emerging toxins potentially found in end-of-life electronics that may be
released by various processing technologies (e.g., brominated flame retardant (BFR) emissions from shredding and
pulverizing plastics). Support transfer of this knowledge to electronics recyclers.
(9) EMERGENCY PREPAREDNESS AND RESPONSE
Awareness of appropriate
management practices for
emergency preparedness
• Creation of best practice guidelines for material-specific spills (e.g., CRT glass and mercury spills) and educate employees
on the dangers of personal exposure to lead and mercury.
• Expanded emergency drill testing. The majority of facilities focus on just fire drills every year. Testing could be rotated to
include mock medical emergency, mock spill, severe weather, etc.
(10) MATERIALS RECOVERY AND DISPOSITION
Awareness of best practices
Verification of downstream
processors
• Improvement of training materials for auditor knowledge of downstream processing options and best practices.
• Update relevant web sites, deliver webinars and/or email updates on tools and guidance to help electronics industry
stakeholders improve their understanding of best practices.
• Provision of auditing tools/ training/ questions to assist auditors to fully explore audit trails for downstream processing.
• Creation of case studies for training materials on how to audit downstream processing.
(11) EXPORT (APPLIES TO Focus MATERIALS (AS DEFINED BY R2) /HAZARDOUS ELECTRONIC WASTES (AS DEFINED BY E-STEWARDS ONLY)
Implementation
training and
awareness
• Create tools such as a lockbox or other confidential means by which auditors can verify information without disclosing
commercial terms or confidential business information (CBI).
• Develop and conduct additional training for auditors on how to evaluate compliance with export requirements, including using
case studies.
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewardsฎ
                             Appendix A- p.3

-------
ISSUE
SUGGESTED STRATEGIES FOR IMPROVEMENT
(12) SITE CLOSURE AND INSURANCE
Strength of closure plans and
financial assurance
requirements; better
understanding of this by
stakeholders
        Develop case studies or recommendations regarding appropriate insurance limits and instruments focusing on site closure
        "financial surety."
(13) DOWNSTREAM ACCOUNTABILITY
Downstream accountability
        Improve auditor and facility understanding of limits of use of assets as closure mechanism.
                                   •   Standardized audit packet guidance tool - what an upstream can ask for from its downstream, as a basic practice in the
                                       electronics recycling industry.
                                   •   Develop easier-to-use and standardized tools or requirements for mass balance reports (e.g., new e-Stewards standard
                                       requires linking shipping records to tracking information).
                                       Tools to keep transparency of downstream vendors but which respect commercial privacy.
                                       Consider audit queries by auditors for downstream sampling to go beyond getting data from electronics recyclers' EH&S
                                       staff (e.g., ask Accounting).
(14) MONITORING AND MEASUREMENT
Effective use of monitoring and
measurement tools and
systems
                                   •   Ensure compliance with legal requirements is transposed into periodic monitoring / measurement tools which are used
                                       regularly.
    •   Standardized monitoring and measurement software for the electronics recycling  industry,  crossing  all operational and
        compliance areas.
                                Implementation Study of the Electronics Recycling Standards: R2 and e-Stewardsฎ
                                                                Appendix A- p.4

-------
ISSUE
SUGGESTED STRATEGIES FOR IMPROVEMENT
(15) TRACKING
Tracking to downstream
destinations - adequate time to
audit
• Ensure auditors have control of sample selection when tracking to downstream destinations (and have adequate time for to
audit).
• Fine-tune formulas for audit time to ensure adequate time for sufficient sample size review and analysis. Formula could
consider the number of downstream vendors, volume, complexity of material, complexity of organization. The Standards
owners could do an analysis of adequacy of audit time - in relation to this specific issue.
(16) EVALUATION OF COMPLIANCE
Understanding how evaluation
of compliance works and
putting it into better practice
• Better enforcement of existing laws by regulators would encourage facilities to "check" their compliance more proactively
and thoroughly.
• See suggested strategies for improvement under (2) System Planning; but also provide training on 'how to do this.'
• Ensure "evaluation of compliance" procedures include looking at export, transportation, and data security - these areas are
often overlooked.
(17) NONCONFORMITY, CORRECTIVE AND PREVENTIVE ACTION
Correct use of and auditing of
nonconformity, corrective and
preventive action tools
• Develop easier-to-use tools for doing Corrective and Preventive Action-related work - e.g., apps for smart phones and
tablets, software for facilities (noting that operational folks are less inclined to use paper and forms).
• Periodic review of nonconformity, corrective and preventive action requirements and best practices for auditors as well as
facilities, perhaps as part of CEU-style requirements.
(18) CONTROL OF RECORDS
Correct control of records
• Develop easier-to-use tools for records control - e.g., apps for smart phones and tablets. Use scheduling tools as part of
improving operations.
(19) INTERNAL AUDITS
Internal audit issues
• Use CEUs as tool to ensure continued training in key topic areas, such as how to effective conduct an internal audit,
qualifications of internal auditors, how to effectively use the findings of an internal audit.
• Offer more frequent training (including training for new areas highlighted above), which should lead to better availability of
skilled internal auditors.
• Clarification of minimum competency requirements for conducting internal audits.
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewardsฎ
                             Appendix A- p.5

-------
ISSUE
(20) MANAGEMENT REVIEW
Management commitment and
company culture
SUGGESTED STRATEGIES FOR IMPROVEMENT
                                  •   Target electronics recycling facilities' management via webinars, conferences, and other professional settings, offering
                                     training specific to management and owners - not just EH&S managers.
                                      Research how managers can change company culture and employee behavior when implementing an EH&S management
                                      system, specifically for the electronics recycling industry; disseminate findings through trainings.
                               Implementation Study of the Electronics Recycling Standards: R2 and e-Stewardsฎ
                                                             Appendix A- p.6

-------
Appendix B: Audit Observation Checklist
                      U.S. EPA Office of Resource Conservation and Recovery
                                Implementation Study of R2 and e-Stewardsฎ

                                      [INSERT STANDARD] [INSERT AUDIT TYPE] Audit
                            [INSERT COMPANY NAME] in [INSERT STATE], [INSERT DATES]
                      Audit Observation Checklist
        Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards4
                               Appendix B - p. 1

-------
1.  BACKGROUND
  This checklist is to be used in conjunction with the requirements of the standard being audited (i.e., R2, e-
Stewards).
  Goal is to ensure that waste electronics are being responsibly managed from an environmental, health, safety and
data security perspective.
  The purpose of observing the audits is to:
  ^ Evaluate the effectiveness of the auditing process and of the entire system -e.g., strengths &
deficiencies
  ^ Identify examples of good management practices.
  ^ Document the findings of the audits
  >• Identify recommendations for improvement, if appropriate
  The result will be an evaluation of both the certification process and effectiveness of the system.
    AUDIT OBSERVATION PROTOCOLS: Role of the audit observer
  DO:
  • Review the Audit Observation Preparation Notes
 >• Oversee audit process in an impartial manner and follow the Audit Observation Protocols
 >• Request documents reviewed by the auditor
 ^ Be aware that the audit is intended to verify conformity to both the standards and the facility's
management system
  DO NOT:	
 >• Ask questions of the auditee during the audit process
 >• Answer questions posed by the auditee
 >• Prejudge an auditor's audit trail
 ^ Lead the CB Auditor or indicate flaws in their process
 ^ Mention the names of other facilities involved in the study
  Confidentiality & Independence
 ^ Verify conflict of interest and confidentiality status of Audit Observer
  Safety
 >• Review beforehand and conform to Gracestone Inc. Health and Safety Policy
 ^ Audit Observers are responsible for their own safety, for taking action to avoid injury, for conforming
to the facility's safety procedures, consulting with the CB lead auditor if there are questions about safety,
and if necessary, leaving the facility.
   Confirm all appropriate security, safety and PPE requirements
         Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards4
                                      Appendix B - p.2

-------
3. AUDIT OBSERVATION PLANNING
Q Is the NDA between EPA & CB in place and is the facility aware the EPA Team will be coming
onsite?
Q Was a copy of previous audit report provided?
Q Was a copy of the audit plan provided?
Q Confirm location, dates, and start time
Q Confirm contacts at the facility, the auditor, ANAB auditor (if applicable), and start time
Q Familiarization of company business model/ Technologies (Shredders, Balers, CRT cleaning),
processed
materials
Q Familiarization of regulatory requirements
4. ROLES
Participant
EPA
ANAB
Electronics
Recycler
being
audited
Certifying
Body
&R
Ro
>C
>v
ป>p
>s
ป>p
ป>c
ป>p
role
ป>0
>>IV
dire
ป>P
ป>P
res
ป>H
ป>IV
>>C
EP/
>>C
>>C
   & RESPONSIBILITIES OF KEY PLAYERS
     Role in Audits for the Implementation Study
      Overall Project Director; up to 3 observers may be onsite at audits

      Will work directly with ANAB and CBs to schedule audits EPA & contractor can observe

      Play observation role during audit and participate in any post-audit discussions

      Schedule pre-audit logistics call with Electronics Recycler being audited
      Provides dates & specifics of two Witness audits for two CBs to EPA
      Coordinates with EPA to schedule selected audits
      Participates in pre-audit logistics calls hosted by EPA, if needed, to go over all details,
    roles and responsibilities for the scheduled Witness Audit(s)

      Oversees all ANAB formal witness audits according to ANAB's policies and procedures

      May attend and observe the additional audits being conducted by CBs that EPA schedules
    directly with  CBs (depending on ANAB's availability/interest)
      Participates in post-audit Q&A sessions
      Participates in pre-audit logistics call hosted by EPA to go over all details, roles and
    responsibilities for the scheduled audit
      Hosts the witness audit
      May participate in post-audit Q&A with EPA.
      CB (and its auditor if deemed necessary) participates in pre-audit logistics call hosted by
    EPA to go over all details, roles and responsibilities for the scheduled audit(s)

    ^ CB Auditor conducts appropriate audit (initial or surveillance, etc.).
    ^ CB may send its own staff person to witness the auditor, if it wishes.
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards4
                              Appendix B - p.3

-------
5. General Information
Audit Observer
ANAB Witness
Auditor
CB Auditor Name(s)
Name, location of
facility
Facility contact
Facility contact
Standard(s)


Audit Date(s)





Q e-Stewards
LI OHSAS 18001

a
Type of audit Stage
I 1
Scope of
certification
QR2
QRIOS
# Audit Days
EPA
Representative
Certification
Body
# of employees
Phone:
Phone:
LI ISO 14001






LI ISO 9001
Q Additional Standards:

J
Stage 2
a 1st Yr
Surveillance
Ll2ndYr
Surveillance
Q Recertification

Functional Units 1 	
Other
6. TYPE OF OPERATIONS / PROCESSING (CHECK ALL THAT APPLY)
Q Reuse: parts
harvesting
Q Refurbishment
Q Data destruction
Q Manual
disassembly
Q Shredding
Q Compacting/Baling
Q Breaking, cutting, crushing, shredding, or pulverizing CRTs
Q Removing mercury-containing components
Q Using power machinery to shred, cut, grind, or shear EE
Q Using a shredder dedicated to hard drives
Q Using thermal processes for melting, smelting, or combustion of Electronic
Q Using acids or solvents for precious metals or plastics recovery or cleaning
Equipment
procedures

7.   TYPE OF MATERIALS PROCESSED (CHECK ALL THAT APPLY)
^^^^i

Q Electronic Equipment, parts, and materials for which the constituents are unknown (get examples, and determine


how this facility and jurisdiction define "EE", including UW classification notes)
   Circuit boards or circuit board bearing material (printers, keyboards, mice, fax machines)
   Printer, toner & toner cartridges, &/or copy drums (containing selenium and/or arsenic)
   Mercury-containing devices (mercury switches, laptops, flat panel monitors)
Q Batteries
Q Leaded display
glass
Q Components containing radioactive substances
Q Items containing polychlorinated biphenyls (PCBs)
   Other, especially any other toxic materials:
          Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards4

                                         Appendix B - p.4

-------
8. ITEMS TO REVIEW AT THE OPENING MEETING
Q Purpose of the observation
Q Evaluate the effectiveness of the auditing process and of the entire system -
e.g., strengths & opportunities for improvement
Q Identify examples of good management practices
Q Document the findings of the audits
Q Identify opportunities for improvement, if appropriate
Q Questions for the facility
Q Please let us know if there any documents you would rather not share e.g.
due to confidentiality
Q Are photographs acceptable?
Q How many employees at the site?
Q How many shipments are picked up from the facility each year?
Q Have there been any process or equipment changes in the past year?
Q Request auditor verbalize audit process for the Audit Observers
Q Identify which clause is being audited
Q Share documents and records (unless they are confidential)
Q Sample sizes for records
Q Audit observers will be following protocols to ensure we do not influence the audit:
Q In essence we are not really here
Q Are not able to give guidance or to ask questions
Q Not make comments
Q Not identify areas of potential non-conformity during the audit
Q Outputs
Q EPA Implementation Study Report
Q No company-specific information will be included
Q Follow-up
Q Interviews with auditor, and EHS Rep and Operations Rep























Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards4
                             Appendix B - p.5

-------
9. AUDIT OBSERVATION
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.1. System requirements
D Scope & EH&S Management
System based on Plan Do Check
Act
D Certified to EH&S
Management System
08:l.(a)(2)
13:l.(a)-(c)
13:l.(b)
V8:4.1
V13:4.1
V8:4.1
V13:4.1


















Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.2. System planning
D EH&S Policy
D Policy/Plan based on
hierarchy of responsible
management
D EA and hazards/Risk
assessment
D Identification of legal / other
requirements EH&S, Permits,
Waste Storage, Labeling,
Storm-water, Export
D Identification of legal / other
requirements procedure
13:l(b)
08:l.(a)(3)(A)
13:2.(a)
08:l.(a)(2),4(c)
13:l.(b),4.(c)
08:l.(a)(l)l.(a)(3)(B)
13:l.(b),3.(a)(l)
08:l.(a)(l)l.(a)(3)(B)&
3.(1)
13:l.(b),3
V8: 4.2
V: 13 4.2
V8: 4.4.6.4
V13: 4.3.4
V8: 4.3.1
V8: 4.3.2
4.3.2.1
4.3.2.2
4.3.2.3
V13: 4.4.2,
4.3.2.1
V8: 4.3.2
4.3.2.1
4.3.2.2
4.3.2.3
V13: 4.3.2













































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                             Appendix B - p. 6

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.2. System planning
D Identification of legal / other
requirements: Data Security
D Identification of legal / other
requirements related to storm-
water
D Identification of legal / other
requirements related to EH&S
including permits, Waste
Storage, Labeling.
D Identification of legal / other
requirements related to export
D EPA: How well are the air
emissions health and safety
requirements being met?
D EPA: Were legal
requirements for shredder dust
reviewed?
D Objectives, targets, & action
plan
D Availability of resources
08:l.(a)(l)l.(a)(3)(B)&
3.(1)
13:3
08:l.(a)(l)l.(a)(3)(B)&
3.(1)
13:l.(b),3
08:l.(a)(l)l.(a)(3)(B)&
3.(1)
13:l.(b),3
08:l.(a)(l)l.(a)(3)(B)&
3.(1)
13:3.(2)


08:l.(a)(2) l.(a)(3)(f)
13:l(b),l(c)(l)
08:l.(a)(2)
13:l(b)
V8: 4.3.2
4.3.2.1
4.3.2.2
4.3.2.3
V13: 4.3.2
V8: 4.3.2
4.3.2.1
4.3.2.2
4.3.2.3
V13: 4.3.2
V8: 4.3.2
4.3.2.1
4.3.2.2
4.3.2.3
V13: 4.3.2
V8: 4.3.2
4.3.2.1
4.3.2.2
4.3.2.3
V13:
4.3.2.1


V8: 4.3.3
V13: 4.3.3
V8: 4.4.1
V13: 4.4.1








































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                             Appendix B - p.7

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.2. System planning
D Roles, responsibilities &
authorities defined,
communicated & understood
D System documentation is
identified
Document control procedure
implemented
08:l.(a)(2)
08:l.(a)(3)(f),4.(d)(2)(A)
08:l.(a)(2)
V8: 4.4.1
V13: 4.4.1
V8: 4.4.4
V13: 4.4.4
V8: 4.4.5
V13: 4.4.5



























Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for anything
less than 5]
9.3. Training and communication
D Training needs identified
D Workforce has appropriate
training - EH&S, data
security, export, etc.
D Communication
08:l.(a)(2),4.(d)(2)(A)
13:l.(b),
4.(d)(2)(A)
08:l.(a)(2),4.(d)
13:l.(b),4.(d)(2)(A)
08:l.(a)(2),4.(d)-(g)
13:l.(b),4.(d)
V8: 4.4.2
V13: 4.4.2
V8: 4.4.2
V13: 4.4.2
V8: 4.4.3
V13: 4.4.3



























Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                             Appendix B - p. 8

-------
Requirement
R2
Clause
e-
Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for anything
less than 5]
9.4. Operational control
D Identification & planning of
operations associated with
significant EAs & H&S risks
D Engineering/administrative/PPE
controls
D Administrative operating
procedures that stipulate operating
criteria
D Procedures for significant
environmental aspects of goods
and services used by the
organization
D Communication: suppliers &
contractors
D Operations are controlled
D Management of change
08:4.(c)
13:l.(b),4.(c)
08:4.(d)
13:4.(d)
08:l.(a)(2)
13:l.(b),4.(d)-(e)
13:l.(b)
08:l.(a)(2)
13:l.(b),4(f)
08:l.(a)(2)
13:l.(b),
4.(d)-(e)
08:1. (a)
13:l.(c)
V8: 4.4.5
V13:
4.4.6.1
V8:
4.4.6.1.g
V13:
4.4.6.1
V8:
4.4.6.a-b
V13:
4.4.6.1
V8: 4.4.6.C
V13: 4.4.6
c
V8: 4.4.6.C
V13:
4.4.6C
V8: 4.4.6
V13: 4.4.6,
4.4.6.3g
V8: 4.4.6
V13: 4.4.6,
4.4.6.3g































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                             Appendix B - p. 9

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.5. Health and safety
D Capability to process EE in a
manner protective of worker &,
public EH&S.
D Workforce, volunteers &
temporary workers in
conformity with system
D Identification of hazardous
substances in the EE
D H&S inspection
D Plan to reduce & eliminate
workplace exposures & physical
hazards
D Ergonomic evaluation
D Industrial Hygiene
monitoring NOTE: EPA is
interested in how well the
housekeeping/
cleanliness criteria (if there are
any) in the standards are being
met.
D Calibration of monitoring
equipment
08:4(a)
13:4.(a)
08:4(f)
13:4.(f)
08:4(c)
13:4ฉ
08:4(b)
13:4.(b)
08:4(d)
13:4.(d)
08:4(c)
13:4.(c)
08:4(e)
08:NA
V8: 4.4.6.1
V13:
4.4.6.1
V8: 4.4.6.1
V13:4.1,
4.4.3. la,
4.4.6.1
V8:
4 .4.6.1. c
V13: 4.3.4
V8:
4.4.6. l.b
V13: 4.4.6d
V8:
4.4.6.1.d
V13:
4.4.6.1
V8:
4 .4.6.1. e
V13: 4.3.1,
4.4.6.1C
V8:
4.4.6.1.1
V13:
4.5.1.2
V8:
4.4.6.1.1
V13:
4.5.1.2








































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 10

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.5. Health and safety
D ISO 17025 certified
lab/equivalent used
D Semi-annual monitoring of
high hazard risk
D Precautionary approach to
risk reduction
D Informing workers of results
D Injury & illness prevention
program
D H&S committee
D Designated H&S coordinator
D Two-way communication on
H&S
D Investigation & resolution of
H&S complaints
NA
08:4.(e)
13:4.(e)
NA
08:4.(e)
13:4.(e), 4.(g)
08:3.(a)(l)
13:3.(a)(l),
4.(d)
08:3.(a)(l)
13:3.(a)(l)
08:4.(g)
13:4.(g)
08:4.(g)
13:4.(g)
NA
V8:
4.4.6.1.1
V13:
4.5.1.2e3
V8:
4.4.6.1.1
V13:
4.5.1.2 &
App A
V8:
4.4.6.1.1
V13:
4.4.6.1
V8:
4.4.6.1.1
V13:
4.4.3. Ia3
V8:
4.4.6.1. l.c
V13:
4.4.6.1
V8:
446 1 Id
V13:
4.4.3. Ia4
V8:NA
V13:NA
V8:
446 1 Id
V13:
4.4.3. Ia4
V8:
4.4.6.1. l.e


















































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 11

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.5. Health and safety

D Ongoing hazards & risks
assessment
D Engineering, administrative,
& PPE controls
D Access to MSDSs
D Workplace hygiene / cleaning
procedures
D Designated occupational
health provider for medical
surveillance
D Medical surveillance

08:4.(c)
13:4.(c)
08:4(d)
13:4.(d)
08:3.(a)(l)
13:3.(a)(l)
08:4(b)
13:4.(b)
NA
08:4(d)(2)(D)
13:4.(d)(2)(D)
V13:
4.4.3. Ia4
V8:
4.4.6. 1.1. f
V13: 4.3.1
V8:
446 1 1 g
V13:
4.4.6.1
V8: 4.4.6
V13: 4.4.6,
4.4.6.3g
V8:
446 1 li
V13:
4.4.6.1b
V8:
44611k
V13:
4.5.1.2e
V8:
44611k
V13:
4.5.1.2e































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 12

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.6. Reuse
D Prohibit sale or donation for
reuse if contrary to commercial
agreements
D Functionality testing
D Implement a Quality
Assurance Plan (or maintain
ISO9001 certification)
D Availability of functionality
test results
D Data eradication procedures
D Labeling
D Accessibility of identifying
records
D Protective packaging of
refurbished equipment
D Assurance that equipment is
destined for Reuse and not
Recycling or Final Disposal
08:6.(a)
13:6.(a)
08:6.(c)(l)
13:6.(c) (1)&(2)
B^OaXB^.OaXB)
08: 13. (a)
08:8.(b)
13:8.(b)
08:6.(b)(l)
08:9.0 & 13.0
08:6.b.2; 12(a)
13:6.(b)(3); 9.(a),12(a)
08:6.(c)(3)(B)
V8:NA
V13:NA
V8:
4.4.6.2.a
V13:
4.4.6.2a
V8:NA
V13:NA
V8:
4.4.6.2.a
V13:
4.4.6.2a
V8:
4.4.6.2.C
V13:
4.4.6.2b
V8:
4.4.6.2.d
V13:
4.4.6.2 c
V8:
4.4.6.2.d
V13:
4.4.6.2c,
4.4.3. Ib
V8:
4.4.6.2.e
V13:
4.4.6.2d
V8:
4.4.6.2.f

















































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 13

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.6. Reuse



D D Brokers reselling tested
working equipment for reuse

D Management of FM/HEW
scrap from reuse

D Mass balance includes
equipment going for reuse

D Take Back Service/Product
Return Plan


D Outsourcing
D Requirements for
"Collectible, Specialty or
Unusual" electronics



NA

08:6.(c)(3)(B)
13:6.(c)(3)(B) (iii)

08:7
13:7.(a)

13:6.(c)(l)(C), 6.(c)(2)(D)


08:6.(c)(3)
13:6.(c)(3)(B)
13:6.(d)
V13:
4.4.6.2e
V8:
4.4.6.2.f.2
V13:
4.4.6.5 a
V8:
4.4.6.2.g
V13:
4.4.6.2g
V8:
4.4.6.2.h
V13:
4.5.1.3
V8:
4.4.6.2.i
V13:
4.4.6.2f
V8:
4A.6.2.J
V13:
4.4.6.2h
V8: 4.4.6.2
Table A
V13:
4.4.6.2 a







































































































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 14

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.7. Data Security/data destruction
D Data destruction according to
NIST 800-88
D Employee training &
evaluation on data destruction
D Review & validation of data
destruction processes by an
independent party
D Quality controls documented
for data destruction
D Communication with
Customers awareness of
concerns over loss of data, and
of service terms & liabilities
D Protection of data from theft
or loss/Security
D Documentation of data
destruction procedures
D Procedures used to address
imperfections in hard drives
D Maintain records of data
destruction
08:8(a)
13:8.(a)
08:NA
13:8.(c)
08:8(d)
13:8.(d)
13:8.(e)
NA
08:10
13:8.(f); 10.
08:8.(b)
13:8.(b),8.(e)
08:NA
13:8.(b),8.(e)
13:8.(g)
V8:
4.4.6.3.b.3

V8:NA
V13:NA
V8:NA
V13:NA
V8:
4.4.6.3.d
V13:
4.4.6.3b
V8: 4.4.6.3
&
4463b 1
V13:
4.4.3. Ib
V8:
4.4.6.3.b.2
V13:
4.4.6.3b&e
V8:
4.4.6.3.b.4
V13:
4.4.6.3d
V8:
4.4.6.3.b.3
V13:
4.4.6.3c
V8:
4.4.6.3.d
V13:
4.4.6.3d

















































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 15

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.7. Data Security/data destruction
D Requirements for
Outsourcing of data destruction
D Provision of information to
customers about liabilities
13:8.(h)
NA
V8:
4.4.6.3.f
V13:
4.4.6.3f
V8:
4.4.6.3.b.5
V13:
4.4.6.3a


















Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding


Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.8. Management of Focus Materials, Hazardous Electronic Wastes, and Problematic Components & Materials
D Plan/Hierarchy for
FMs/HEW/PCMs
D Management of Focus
Materials/ Hazardous e-Waste /
Problematic Components &
Materials
D Safe removal of
FMs/HEW/PCMs (see details in
the standard)
D Separation and storage
D EPA: Were notifications, and
requirements about labeling, if
they are shipping to overseas
08:2.0
13:2.0; 5.(a)
08:5.0
13:5.(a)
08:5.0
13:5.(b)
08:5.(b); 9.(a)
13:5.(b),9.0

V8: 4.4.6.4
V13: 4.3.4
V8: 4.4.6.4
V13:
4.4.6.4
V8:
4.4.6.4.a
V13:
4.4.6.4a
V8:
4.4.6.4.b
V13:
4.4.6.4d &
e














































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 16

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding


Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.8. Management of Focus Materials, Hazardous Electronic Wastes, and Problematic Components & Materials
smelters for precious metals
recovery reviewed?
D Limited storage time for
FM/s/HEW
D EPA: Can recycler
demonstrate that CRTs are being
managed in a compliant fashion,
either onsite or at their
downstream vendor
D Storage for reuse
l~l EPA: For facilities that shred,
is the auditor asking for TCLP
results?

08:3.(a)(l)
13:3.(a)(l), 9.(a)(2)

08:9.(a)
13:9. (a)


V8:
4.4.6.4.C
V13:
4.4.6.4e

V8: 4.4.6.4
V13:
4.4.6.2d
















































R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
O
FI
N
C
Audit Observation
1
2
3
4
5
Comments/Evidence/Justific
ation
[Comments required for
anything less than 5]
9.9. Emergency preparedness & response
D Up-to-date Emergency
preparedness & response
procedure(s)
D Periodic review & revision of
procedures
D Periodic testing of procedures
08:l.(a)(3)(E), l.(2)
13:4.(h)
08:l.(a)(3)(E), l.(a)(2)
13:l(c)
08:NA
13:4.(h)
V8: 4.4.7
V13: 4.4.7
V8: 4.4.7
V13: 4.4.7
V8: 4.4.7
V13: 4.4.7



























Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 17

-------
Requirement
R2
Clause
9.10. Materials recovery & disposition
D Treatment and control of final

disposition of FMs, HEW and
PCMs

l~l Only disposed of in licensed
and permitted facilities

D Not disposed of in solid waste
landfills or incinerators

throughout the Recycling Chain
D Controls for recycling of
plastics and resin materials
containing or consisting of
Halogenated Compounds

D Control of waste /residues


D Knowledge and approval of
final disposition BEFORE
shipment.


D Equipment is packaged
appropriate to risks during
transportation to public health or
the environment and level of care

e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
N
C
Audit Observation
1
2
3
4

08:5.(a)

08:5.(c)
13:5.(c)(d) & (e)(3)





08:l;2.(a)(3);5.(d)
13:5.(d)
NA
08:3.(a)(2)
13:2.(a)(3); 5(d)
08:3.(a)(2)
13:3.(a)(2)
08:12.(b)
V8: 4.4.6.6

V13:
4.4.6.6
V8: 4.4.6.6
V13:
4.4.6.6
V13:
V8: 4.4.6.6
V13:

4.4.6.6
V8- 4466
V13:
4466

V8: 4.4.6.6
V13:
4.4.6.6 &
Def3.42
V8- 4466
V13:
4.4.6.6 &
4.4.6.5
V8:
4467
V8-

4465
V13:
4.4.6.4


































































































































































5
Comments/Evidence/Justifica
tion
[Comments required for
anything less than 5]






























































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 18

-------
Requirement
R2 e-Stewards
Clause Clause
Other
(NA)
Clause
CB
Finding
OFI
N
C
Audit Observation
1
2
3
4
9.10. Materials recovery & disposition
D EPA question: written
documentation that their
transporters have regulatory
authorizations and no significant
violations
D Management of ink and toner
cartridges
08:12.(b)
13:5(h)
V8:NA
V8:
4.4.6.6.g
V13:
4.4.6.6f












5
Comments/Evidence/Justifica
tion
[Comments required for
anything less than 5]







Requirement
R2
Clause
e-Stewards
Clause
Oth
er
(NA
)
Cla
use
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
9.11. Export restrictions for focus materials (R2) and Hazardous Electronic Waste (e-Stewards)
D Import & Export controls in
place through to Final Disposition
including Intermediaries
D Export of cleaned CRT cullet
only:
D D If it is thoroughly cleaned
D D Will be used as a direct
feedstock
l~l Competent Authority (export,
transit & import ) countries in file
08:5.(e)(4)
13:3(a)(2), 5.(e)(3) & (4)
08:3.(a)(2)(B)
13:3.(a)(2)
V8: 4.4.6.7
V13:
4.3.2.1 &
4.4.6.7
3.4.4, V8:
4.4.6.7.C
V13:
4.4.6.7 a2












5
Comments/Evidence/Justificati
on
[Comments required for
anything less than 5]







Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            AppendixB -p. 19

-------
Requirement
R2
Clause
e-
Stewards
Clause
9.12. Site closure & insurance
D Site closure plan
D Plan assures proper closure of
the facility & against
abandonment of EE
D Plan includes indoor dust
sampling where PHPTs have been
used indoors
D Ensures remediation of any
contamination for workplace dust
D Includes facility soil and
groundwater testing for threshold
levels listed in 3.43(b) if PHPTs
are used or HEE or PCMs are
stored or managed outside of fully
contained, impermeably floored
buildings
n Stipulates testing be conducted
by a third party including a signed
affidavit
D Legal and financial assurances
for proper closure
D Insurance for potential risks
and liabilities (environmental
risks)
08:ll.(b)
13:ll.(b)
08:ll.(b)
13:ll.(b)
13:ll.(b)(3)
13:ll.(b)(3)
NA
NA
08:ll.(b)
13:ll.(a)&(b)
08:ll.(b)
13:ll.(a)&(b)
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4

V8: 4.4.6.8
V13: 4.4.9
V8: 4.4.6.8
V13: 4.4.9
V8: 4.4.6.8
V13: 4.4.9
V8: 4.4.6.8
V13: 4.4.9
V8: 4.4.6.8
V13: 4.4.9
V8: 4.4.6.8
V13: 4.4.9
V8: 4.4.8
V13: 4.4.9
V8: 4.4.8
V13: 4.4.8
















































5
Comments/Evidence/Justificatio
n
[Comments required for
anything less than 5]

























Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 20

-------
Requirement
R2
Clause
e-
Stewards
Clause
9.12. Site closure & insurance
D Insurance for potential risks
and liabilities (data destruction
risks)
D Insurance in place to
underwrite it if indemnification is
offered
D Clear specification to
customers of what
indemnification are or are not
being offered
D Utilize PHPTs? Pollution
Liability insurance is
commensurate with the nature/
size of the operation
NA
NA
NA
08:ll.(b)
13:ll.(a)
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4

V8: 4.4.8
V13:4.8
V8: 4.4.8.b
V13: 4.4.8
V8: 4.4.8.a
V13:
4.4.3. Ib
V8: 4.4.8.C
V13: 4.4.8
























5
Comments/Evidence/Justificatio
n
[Comments required for
anything less than 5]













Requirement
R2
Clause
e-
Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
9.13. Downstream accountability
D Downstream vendor flow chart
D Vendor selection
D Initial due diligence (per
standard requirements)
13:7.(b)
08:5.(e)
13:5.(e)
08:5.(e)
13:5.(e)
V8:
4.5.1. l.d
V13:
4.4.6.5a
V8: 4.4.6.5
V13:
4.4.6.5
V8: 4.4.6.5
V13:
4.4.6.5


















5
Comments/Evidence/Justificatio
n
[Comments required for
anything less than 5]










Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 21

-------
Requirement
R2
Clause
e-
Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
9.13. Downstream accountability
D Ongoing due diligence (per
standard requirements)
D Shipping records
D Did the auditor look at
documentation for shipments and
check record-
keeping requirements (for R2)?
D Downstream audits records
D Outsourcing of Reuse tasks
(next tier only)
D Responsibility for equipment
going for reuse until it is sold or
donated as fully functional
equipment or parts
08:5.(f)
13:5.(f)
08:5.(e)(7);3.(a)(2);13(a)
13:5.(e)(6), 3.(a)(2); 13.(a)

08:5.(f);13.(a)
13:5(e),5(f); 13.(a)
08:NA
13:6.(c)(3)
NA
V8: 4.4.6.5
V13:
4.4.6.5
V8: 4.4.6.5
V13:
4.4.6.5

V8: 4.4.6.5
V13:
4.4.6.5
V8: 4.4.6.2
V13:
4.4.6.2h
V8: 4.4.6.5
V13:
4.4.6.2




































5
Comments/Evidence/Justificatio
n
[Comments required for
anything less than 5]



















Requirement
R2
Clause
e-
Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justificatio
n
[Comments required for
anything less than 5]
9.14. Monitoring and measurement
D Procedure/Plan to monitor and
measure operations that could
have a significant EH&S impact?
08:l.(a)(2)
13: l(b)
V8: 4.5.1









Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 22

-------
Requirement
R2
Clause
e-
Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justificatio
n
[Comments required for
anything less than 5]
9.14. Monitoring and measurement
D Monitoring of performance of
applicable operational controls
D Monitoring of EH&S
objectives and targets
D Calibration - schedule &
records
08:l.(a)(2)
13:l.(b);4(e)
08:l.(a)(2)
13:l.(b),l.(c)
08:l.(a)(2)
13:l.(b),6.(c)(l)(B)&
(2)(B)
V8: 4.5.1
V8: 4.5.1
V8: 4.5.1



























Requirement

R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.15. Tracking
D Tracking system (controlling,
weighing, & documenting) total
incoming & outgoing materials,
wastes, & equipment &
components going for reuse,
including off-site storage
D Calculation of a Mass Balance
Accounting on a six-monthly
basis at a minimum & linking of
shipping records
D Does the mass balance

calculation include equipment
going for reuse



08:7.0
13:7.0



NA




NA



V8' 4511
V13:
4513


V8:
4.5.1. La
V13:
4 5 1 3b &
c
V8:
4 5 1 1 a
V13:

4.5.1.3b







































































































































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 23

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.15. Tracking
D Monitoring and control of the
destinations of HEWs
D Provision to customers (if
requested) of records and contact
info of Downstream Recyclers
throughout the Recycling Chain
D Provision to customers (if
requested) of downstream flow
chart
D Accurately quantify, record &
submit data to the central
database on an annual basis, for
all certified sites (i.e. mass
balance, test results etc)
08:5.(e)(7)
13:5.(e)(6)
13:7.(b)
13:7.(b)
NA
V8: 4.5.1.1
V13:
4.5.1.3
V8:
4.5.1. l.c
V13:
4.4.3. Ib
V8:
4.5.1. l.c
V13:
4.4.3. Ib
V8: 4.5.1.2
V13:
4.5.1.4




































Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
9.16. Evaluation of compliance
D Procedure/plan for evaluating
compliance with EH&S legal &
other requirements
D Procedure for evaluating
compliance with export legal &
other requirements
08:3.(a)(l)
13:l.(b),3.0
08:3.(a)(l)
13:l.(b);3.(a)(2)
V8: 4.5.2
V13: 4.5.2
V8: 4.3.2.2
V13:
4.3.2.1 &
4.5.2












5
Comments/Evidence/Justification
[Comments required for
anything less than 5]







Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 24

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justification
[Comments required for
anything less than 5]
9.17. Nonconformity, corrective & preventive action
D Nonconformity, Corrective
Action and Preventive Action
Procedure
D Review of the effectiveness of
corrective action(s) and
preventive action(s) taken
D Nonconformities, corrective &
preventive actions lead to EH&S
MS change
08:NA
13:l.(b)
08:NA
13:l.(b)
08:l.(a)(2)
13:l.(b)
V8: 4.5.3
V13:
4.5.3
V8:
4.5.3.e
V13:
4.5.3.e
V8: 4.5.3
V13:
4.5.3



























Requirement
9.18. Control of records
R2
Clause
e-
Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justificatio
n
[Comments required for
anything less than 5]

D Records demonstrate flow of
materials
D Records demonstrate system
conformity and are maintained in
a single location
D Records Control Procedure(s):
ID, storage, protection, retrieval,
retention (>5 years) + 30 years for
medical records (e-Stewards)
D Records are controlled, backed-
up, disposed of according to
procedure
08:7
13:7.0
08:13
13:13.0
08:NA
13:l.(b)
08:NA
13:l.(b)
V8:
4.5.1.1
V13:
4.4.6.5 c 3
&d
V8: 4.5.4
V13:NA
V8: 4.5.4
V13:
4.5.4.1
V8: 4.5.4
V13: 4.5.4




































Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 25

-------
Requirement
R2
Clause
e-Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
9.19. Internal audit
D Annual system review /
periodic internal EH&S system
audits
D Audit program and schedule
D Auditors are competent,
objective & impartial
08:l.(a)(2)
13:l.(b)(c)
08:NA
13:l.(b)(c)
13:l.(b)
V8: 4.5.5
V13: 4.5.5
V8: 4.5.5
V13: 4.5.5
V8: 4.5.5
V13: 4.5.5


















5
Comments/Evidence/
Justification
[Comments required for
anything less than 5]










Requirement
R2
Clause
e-
Stewards
Clause
Other
(NA)
Clause
CB
Finding
OFI
NC
Audit Observation
1
2
3
4
5
Comments/Evidence/Justificatio
n
[Comments required for
anything less than 5]
9.20. Management review
D Management review covering
all elements conducted on a
regular basis and records are
maintained

08:l.(a)(2)
13:l.(b)


V8: 4.6
V13:4.6





































10. Post- Audit Discussion Questions for the Facility Owner/Manager

D How well do you believe the audits are evaluating your
conformance to the Standard(s)?
D How well do you believe the management system is facilitating
continual improvement in your organization?
D How effective do you feel the Standard(s) is/are at preventing
illegal exports?
1



2



3



4



5




>•
>•
*•
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 26

-------
10. Post- Audit Discussion Questions for the Facility Owner/Manager

D How effective do you feel the audit process is at preventing illegal
exports of scrap material for recycling?
D How effective do you feel the audit process is at preventing illegal
exports of scrap material for shipped as "reuse"?
D How effective do you feel the Standard(s) is/are at protecting
worker health and safety?
D How effective do you feel the Standard(s) is/are at securing and
properly handling data bearing devices?
D Was there a particular area of the standard that you believe was not
audited sufficiently in the audit?
D Was there a particular area of the standard that that you believe
was over-audited (spent more time than necessary)?
D Do you believe that the auditor struck a good balance on
evaluation of intent, implementation and effectiveness?
D Do you believe the audit time is appropriate?
1








2








3








4








5









*•
*•
>•
*•
*•
>•
>•
*•
Are there any other comments or thoughts you would like to share with EPA?
11. Post-Audit Discussion Questions for the Facility Operations Staff

D How well do you believe the audits are evaluating your
conformance to the Standard(s)?
D How well do you believe the management system is facilitating
continual improvement in your organization?
1


2


3


4


5



>•
*•
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 27

-------
11. Post-Audit Discussion Questions for the Facility Operations Staff

D How effective do you feel the Standard(s) is/are at preventing
illegal exports?
D How effective do you feel the audit process is at preventing illegal
exports of scrap material for recycling?
D How effective do you feel the audit process is at preventing illegal
exports of scrap material for shipped as "reuse"?
D How effective do you feel the Standard(s) is/are at protecting
worker health and safety?
D How effective do you feel the Standard(s) is/are at securing and
properly handling data bearing devices?
D Was there a particular area of the standard that you believe was not
audited sufficiently in the audit?
D Was there a particular area of the standard that that you believe
was over-audited (spent more time in than necessary)?
D Do you believe that the auditor struck a good balance on evaluation
of intent, implementation and effectiveness?
D Do you believe the audit time is appropriate?
1









2









3









4









5










>•
*•
*•
>•
*•
>•
*•
*•
>•
Are there any other comments or thoughts you would like to share with EPA?
12. Post-Audit Discussion Questions for the Certifying Body Auditors

D How effective do you feel the Standard(s) is/are at preventing
illegal exports of scrap material for recycling?
1

2

3

4

5


>•
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 28

-------
12. Post-Audit Discussion Questions for the Certifying Body Auditors

D How effective do you feel the Standard(s) is/are at preventing
illegal exports of scrap material for shipped as "reuse"?
D How effective do you feel the Standard(s) is/are at protecting
worker health and safety?
D How effective do you feel the Standard(s) is/are at securing and
properly handling data bearing devices?
D Are there any areas of the Standard(s) you feel the Standard
Owners (BAN or SERI) could provide better training for auditors?
D Do you believe you had enough time to audit the standards
effectively?
1





2





3





4





5






>•
>•
>•
>•
>•
D Are there any other thoughts or comments you wish to share with EPA?
D If needed, how did you determine conformance with "abc"? Did you review materials electronically or otherwise, ahead of time, if so, what were they?
13. Post- Audit Discussion Questions for the EPA Audit Observers

Q Do you believe there was adequate time to conduct the audits?
Q Did you feel that the auditor spent adequate time to explore the
requirements?
Q Were there any clauses that you felt were particularly well audited?
Q Were there any clauses that you felt were inadequately audited?
1




2




3




4




5





*•
>•
>•
*•
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p. 29

-------
13.    Post-Audit Discussion Questions for the EPA Audit Observers
Q Do you believe that sample sizes were adequate to verify conformity? E.g.
records and number of people interviewed?
Q Do you believe that the ratio of time spent with the EHS representative and
facility management and employees was appropriate?
 *• Are there any other thoughts or comments?
14. CB's Observations of Facility's EH&S Management System Strengths (In context of Audit Process (i.e., does the audit process help reinforce
strengths?)
15. CB's Observations of Facility's EH&S Management System non conformities (In context of Audit Process (i.e., does the audit process help
reinforce strengths?))
16.  CB's Observations of Facility's EH&S Best Practices
         In context of Audit Process (i.e., does audit help facility learn, implement, & improve best practices?)
                               Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                                                             Appendix B - p. 30

-------
17.  Observations of Audit/certification Process Opportunities for Improvement in Context of Audit Process
         (i.e., are opportunities identified with regard to the implementation of standards?)
18. Evaluation of Effectiveness of the audit in determining that the EH&S Management system...
^ Conforms to requirements identified in the audit scope
*• Demonstrates that non-conformities from previous audits were effectively closed (surveillance audits only)
*• Complies with legal and other requirements
*• Supports pollution prevention
^ Supports a safe work environment
^ Maximizes reuse of equipment
^ Protects and correctly handles data-bearing devices
^ Prevents non-functional equipment being shipped under the guise of "Reusable" equipment
*• Properly controls export of electronics containing hazardous materials to developing countries
1









2









3









4









5









19. Evaluation of Effectiveness of Audit Processes
^ Opening meeting
^ Management of the audit, including adjustments to the audit plan
^ Time management and attention to evaluation of key requirements
*• Audit team's knowledge of the standard(s)
*• Audit team's knowledge of the industry
*• Following of audit trails (keep in mind that not all audit trails can be followed)
^ Communication of findings with auditee
^ Documentation of findings
^ Closing meeting
1









2









3









4









5









                              Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                                                         AppendixB -p.31

-------
20.
Audit Observation Follow-up

Q Conduct follow-up with facility, if not able to complete on-site - Question 9.

Conduct follow-up with certification body auditor, if not able to complete on-site - Question
10.
Request copy of the CB auditor (and ANAB audit, if applicable) report.
Additional considerations to support the assessment (for EPA Audit Observer use only)
D What kind of documentation was reviewed for export and import (do they have documentation of
importation (legality - does the country say it is legal to import) and how is it managed in the
importing country)?
D What was the sample size of the documents the auditor is asking for? (100 shipments go out a year,
& auditor asks 10 documents = 10% sample size)
D Are exports of CRTs in compliance with CRT Final Rule - Specifically, for filing an
Acknowledgement of Consent to export unprocessed CRTs from the US to Mexico for recycling?
(Unprocessed means CRTs that have not had the phosphor coatings removed). Did the auditor check to
make sure the downstream processor is one of the companies currently approved?
D Number of vendors reviewed for due diligence? (Should be 100% of the downstream
recyclers/processors of focus material/hazardous electronic waste.) What documents are viewed to
determine how materials are handled through final disposition?
D What was the sample size of employees interviewed by the CB auditor? How many people work at
the facility?
*•
>•
>•
*•
>•
Implementation Study of the Electronics Recycling Standards: R2 and e-Stewards11
                            Appendix B - p.32

-------