United States
                    Environmental Protection
                    Agency
Atmospheric Sciences Research
Laboratory
Research Triangle Park NC 27711
                     Research and Development
EPA-600/S3-84-105  Dec. 1984
&EPA         Project Summary
                    A Comparative  Evaluation  of
                    GC/MS  Data  Analysis
                    Processing

                    E. D. Pellizzari, T. Hartwell, and J. Crowder
                      Mass spectra obtained by fused silica
                     capillary gas chromatography/mass
                     spectrometry/data system analysis of
                     mixtures of organic chemicals adsorbed
                     on Tenax GC cartridges were subjected
                     to manual and automated interpretative
                     techniques.  Synthetic mixtures  (85
                     chemicals representing 15  chemical
                     classes) were prepared to simulate the
                     following design effects: (1) the occur-
                     rence of similar and dissimilar overlap-
                     ping mass spectra from two or more
                     constituents  present in unresolved gas
                     chromatographic peaks; (2) the occur-
                     rence of similar and dissimilar propor-
                     tions (concentrations) of unresolved
                     components  in  gas chromatographic
                     peaks, and (3) the presence of different
                     chemical classes in a mixture. Environ-
                     mental samples from  seven different
                     geographical areas in  the continental
                     United States were collected and ana-
                     lyzed.
                      Using synthetic mixtures,  the inter-
                     pretative methods evaluated for accu-
                     racy were (1) manual (skilled interpre-
                     ter), (2) INCOS data software, (3) Mass
                     Spectra Fourier Transformed/Search
                     software, and (4) a Research Triangle
                     Institute Mass Spectral Search System.
                     A Rindfleisch deconvolution program
                     was also applied to raw data prior to
                     using the automated procedures. Only
                     manual, INCOS, and the  Research
                     Triangle Institute system were evalu-
                     ated with environmental samples. The
                     deconvolution program was also part of
                     this evaluation.
                      This Project Summary was developed
                     by  EPA's Atmospheric Sciences Re-
                     search Laboratory, Research Triangle
                     Park. NC. to  announce key findings of
the research project that is fully docu-
mented in a separate report of the same
title (see Project Report  ordering in-
formation at back).

Introduction
  The problem of monitoring the envi-
ronment for the presence of potentially
hazardous  agents and assessing their
capabilities to cause health and ecological
effects is enormous. In the United States
alone, 30,000 to 50,000 chemical sub-
stances are produced each year. There
are now well over  4 million chemical
substances registered with the American
Chemical Society Abstract  Service, and
approximately 1,000 new chemicals are
developed  by industry  and placed in
commercial usage annually. In terms of
volume, the production of organic chemi-
cals in the noncommunist world increased
from 7 million tons in 1 962  to 63 million
tons in 1970 and is predicted to increase
250 million tons by 1985.
  Analytical techniques have been devel-
oped for simultaneous analysis of several
hundred vapor-phase organics in an
ambient air sample. One example employs
high  resolution gas chromatography/
mass  spectrometry/data system (GC/
MS/DS), an approach that gives qualita-
tive and quantitative information. How-
ever,  this  technique produces  2,000-
3,000 spectra per sample, which is a
significant number for qualitative inter-
pretation by an experienced investigator.
  A presumably efficient  method  for
interpreting GC/MS/DS data is the use
of computerized spectral searching sys-
tems. Computerized  systems automated
to various degrees have not been exten-
sively compared to manual interpretation

-------
and, thus,  their accuracy is not well
documented. To determine the accuracy
of software interpretative techniques, a
comparative evaluation of GC/MS/DS
data processing techniques was made.
  The primary objective was to synthesize
chemical mixtures of known composition
that would adequately test the accuracy
of mass spectral software identification
systems developed by commercial sources
and at universities. A specific aim of this
research was to determine the effects, if
any, of overlapping spectra, concentra-
tion, and compound classes on the accu-
racy of  the identification by software
algorithms for various components in a
mixture. By testing algorithms with known
authentic chemical compositions, the
accuracy of identification routines was
assessed. Thus, the  initial  evaluation
employed synthetic mixtures and subse-
quent evaluation employed ambient air
samples.
  Three  mass spectral  identification
search routines  were tested with raw
mass spectral data  and with the same
spectral  data that had been processed
through  a deconvolution program. The
deconvolution program,  developed by
Rindfleisch, was designed  to  resolve
overlapping mass spectra that are present
as  coeluting compounds  in a GC peak.
The three  mass  spectral identification
search routines that were tested were (1)
the INCOS  data software supplied on a
Data  General computer (in  this case
interfaced to a Finnigan 3300 GC/MS, (2)
a  Fourier transform analysis  software
(MSFS) routine developed by T. Isenhour
coupled  with a standard U.S. Environ-
mental Protection Agency mass spectral
library, and (3) the Mass Spectral Search
Identification Software (RTI/MSSS) rou-
tine developed at Research Triangle I nsti-
tute. Raw mass spectral data were also
interpreted by a skilled investigator who
had no knowledge of sample composition.
  To furthertest the computer algorithms'
ability to identify organic compounds,
environmental ambient air samples were
collected. Even though synthetic mixtures
were used to evaluate the search algo-
rithms, they do not necessarily represent
the level of complexity  that might be
encountered with environmental samples
because environmental samples may
contain several hundred constituents in
various concentration  ratios. However,
the synthetic mixtures  allowed  a true
method  of assessing the accuracy  of
identification because the composition of
the mixtures was exactly known. With
environmental samples, the composition
is totally unknown and, thus, the compar-
ison of identification was to manual
interpretation. The two algorithms that
were  further examined were the RTI/
MSSS and the INCOS system. (The MSFS
was not included in this  portion of the
study  since it performed so poorly with
synthetic mixtures.)

Procedure
  Compounds representing  alkanes,  al-
cohols, aldehydes, esters, ethers, ketones,
nitriles, nitrogen-containing  compounds,
aromatics, halogenated alkanes, halogen-
ated aromatics, aromatic  aldehydes,
sulfur-containing compounds, acids, and
phenols were selected because they have
been identified in ambient air throughout
the continental United States and are
potentially toxic. Also, these chemicals in
many cases are only partially resolved as
GC peaks and in a few cases  have similar
mass spectra.
  Thus,  this  set of  chemicals has the
desired design characteristics of providing
overlapping  spectra in unresolved GC
peaks and compound class effects on the
quality of chromatographic peaks, both of
which affect the quality of mass spectra
used to evaluate automated interpretative
methods.
  The criteria used for selecting sites
were (1) the chemical classes potentially
emitted because of the particular indus-
trial activities, (2) if the potential emission
rates  were of sufficient  magnitude to
provide significant measurable levels in
the ambient air, (3) the assessibility of
sites to locating sampling systems, (4) the
presence of dense populations near the
chemical industry, and (5) the potential
for a unique background or interferences
to the qualitative analysis by GC/MS/DS.
  The individual chemicals  were chro-
matographed on a 25-m bonded phase
fused silica capillary column.  Using reten-
tion time data,  the chemicals were ar-
ranged in order of resolution  and grouped
according to similar retention times. From
these data,  mixtures  of appropriate
chemicals were  loaded onto Tenax GC
cartridges for analysis by GC/MS. The
mixtures were designed to evaluate the
effect of (1) similar and dissimilar over-
lapping mass  spectra from partially
resolved GC peaks, (2) similar and highly
disproportionate concentrations of unre-
solved chemicals, and (3) different chem-
ical classes present  with  different
chromatographic peak quality.
  Nineteen Tenax GC cartridges were
loaded with  chemicals in various com-
binations (L50-200  ng; M:200-650 ng;
1-1:650-1500  ng; HS:1500-5000 ng).
Cartridges were analyzed  using a Finni-
gan  3300 GC/MS equipped with an
INCOS data acquisition system. The same
fused  silica capillary column and  GC
conditions employed for  retention time
determinations were used for GC/MS
analysis. All GC/MS data acquired by the
INCOS system were processed on hard
disk and archived on magnetic tapes.
  The GC/MS analysis was conducted
within  24  h of preparing  synthetic mix-
tures; the entire set of cartridges required
two and one-half days to analyze.
  The  sampling  and analysis methods
employed  in this study for  environmental
samples were developed by other  re-
searchers. Three samples  were collected
in triplicate. Two locations were used at
each site; one of these locations was used
to collect a high and a low volume sample
(25 and 5 liter, respectively) over a period
of approximately 25 min. Locations were
always  selected  using  meteorological
considerations so that each location was
generally  downwind from the industrial
facility.
   The  identification algorithms used on
the raw data and resulting data from the
Rindfleisch cleanup algorithm for identi-
fication of chemicals in  the synthetic
mixtures were (1) manual interpretation
(composition was unknown to interpre-
ter), (2) the RTI/MSSS, (3) the MSFS, and
(4) the INCOS system.

   The  use of  the INCOS software for
identification of chemicals employed the
chromatographic peak-top enhancement
algorithm, a form of spectral deconvolu-
tion. The operator decided the location of
the peak-top by inspecting the ion chro-
matograms and mass spectra in concert
with the peak-top enhancement routine.
Thus, the operator, not  the  computer,
selected the mass spectrum to be sub-
mitted to  the INCOS search  algorithm.
Except for the manual method that com-
pared  mass spectra to the  Eight Peak
Index  and Wiley  Library and did  not
employ knowledge of  retention  times,
each  of the algorithms was  compared
with and without the Rindfleisch decon-
volution system.

   Mass spectra obtained from the 12
environmental samples were submitted
to (1) manual  interpretation,  (2)  INCOS
software,  and (3) the RTI/MSSS. Statisti-
cal analysis was  performed for mass
spectral data of synthetic and environ-
mental samples to test the comparability
of the different identification routines of
compound identification  and  to identify
which factors (e.g., Rindfleisch deconvolu-
tion), if any, affect their comparability.

-------
Results and Discussion
  The GC resolution was inadequate for
the synthetic mixtures with many consti-
tuents partially resolved from  one an-
other, a desirable condition for this study.
The DB-1 fused silica capillary column
yielded symmetrical peaks with minimal
tailing for most chemicals except the
strong acids and bases.
  Except  for manual interpretation by
skilled researchers, the computer  algo-
rithms provided a ranking value with each
analyte's  identity in a sample.  The five
highest-ranking values were examined to
determine whether the correct identity
was present as one of these choices. The
data (and rankings) were sorted to gam
insight into the accuracy of identification
of these  chemicals by  automated and
manual methods.
  Table 1 presents the overall percent
correct identities by the method of identi-
fication (includes data from all mixtures).
These results  clearly indicate  that the
highest percent accuracy was  obtained
with manual interpretation and the INCOS
algorithm. The poorest results were ob-
tained with  the Fourier transform  mass
spectral search algorithm, which achieved
only 49% correct identifications. On an
overall  basis,  the  application of the
Rindfleisch  deconvolution  algorithm to
raw data prior to subjecting the  mass
spectra to the various identification rou-
tines appeared to have very little effect on
increasing the percent of correctly identi-
fied chemicals.
  Table 2 presents  a summary  of these
design effects on the accuracy of identifi-
cation. The results in Table 2 demonstrate
that manual interpretation and the INCOS
algorithm appeared to give the highest
percent correct identification; the worst
percent correct identification was ob-
tained with  MSFS. For  a few chemical
classes,  the  application of Rindfleisch
deconvolution to raw data appeared to
improve accuracy of identification. How-
ever, in most cases, there was no signifi-
cant improvement. As anticipated, higher
percent identification accuracy  was ob-
tained when  the mass spectra  were
dissimilar for constituents in unresolved
chromatographic peaks.  An exception to
this trend was the results obtained with
the MSFS. Finally, for those  chromato-
graphic peaks representing triplets, quar-
tets, quintets, and sextets, the percent
accuracy was greater when the constitu-
ents were at  approximately equivalent
concentrations than when they appeared
in combinations of high-low concentra-
tions.
 Table  1.    Overall Percent Correct Identification by Method of Identification
 Method
                                   2/-2
                                             3/-3
                                                      4/-4
                                              5/-S    Missing
 Manual

 RTI/MSSS
  -D*
  +D"

 MSFS
  -D
  +D
29.3
20.9
26.9
42.4
41.9
                          70.7
59.0
609
49.5
48.4
6.0
75
3.3
5.0
2.4
2.8
2.3
2.8
1.1
1.5
0.9
0.7
0.5
0.5
1.5
1.3
0
0
0
0
INCOS
-D
+D

21.9
25.4

75.2
58.2

1.3
2.0

0.8
0.9

0.3
0.2

0.5
1.1

0
13.1
 "Without deconvolution.
 ''With deconvolution.


Table 2.    Summary of Design Effect on Percent Accuracy of Identification
RTI/MSSS
Design Effect
Similar Spectra/ Medium Level/
Doublet (N=1 8>c
Similar Spectra/Hi-Low Level/
Doublet (N=36)
Dissimilar Spectra/ Medium
Level Doublet (N=38)
Dissimilar Spectra/Hi-Low
Level Doublet (N=76)
Medium Level/Triplet (N=50)
Hi-Low Level/Triplet (N=100)
Medium Level/ Quartet (N=57)
Hi-Low Level/Quartet 
-------
failed more frequently to correctly identify
the chemicals. As previously indicated,
these results also indicate that the MSFS
had extreme difficulty in properly identi-
fying any of the alkanes.
  An overall comparison was also made
of the percent correct and incorrect
identifications  between  manual and
computer algorithm methods. The com-
bination  of  manual identification and
INCOS algorithm on  raw data (nonde-
convoluted) yielded the highest  agree-
ment (63.6%), i.e., both methods correctly
identified the same chemical. The lowest
percentage agreement observed was for
manual and deconvoluted data using the
MSFS (43.0%).
  A comparison was also made between
computer identification  algorithms  to
determine the percent that the methods
gave either correct or incorrect results.
The  highest agreement  (55.3%) was
between  INCOS (nondeconvoluted data)
and  RTI/MSSS (deconvoluted data).
Similarly, the worst agreement was with
MSFS. Finally,  a comparison between
raw data (nondeconvoluted)  and decon-
voluted data using the INCOS algorithm
was also performed.  Surprisingly, the
agreement when both correctly identified
the analytes was only 61.6%.
  Because  the  correct answer on the
identification of the  chemicals  in the
environmental ambient air samples could
not be known, the only comparison that
could be made  was relative to manual
interpretation. The  greatest percent
agreement occurred between manual and
deconvoluted data interpreted  by the
INCOS system. However, this agreement
was only 53.9%. The percent agreement
by chemical class was also determined;
however, the number of observations in
some cases was rather small. In general,
the INCOS  automated method  agreed
more often by chemical class with manual
than RTI/MSSS.

Conclusions and
Recommendations
  An evaluation of the effect of similar
versus dissimilar mass spectra occurring
in an unresolved chromatographic peak
revealed that a higher accuracy of identi-
fication occurred when the mass spectra
were dissimilar. Deconvolution applied to
raw data did not appear to  improve the
accuracy of identification for the auto-
mated procedures, whether or not the
spectra were similar.
  As the level of chemical increased (50
ng to 5,000 ng/cartridge), the accuracy of
identification also increased until the
mass spectra  became  saturated. The
accuracy decreased after this point. The
saturation effects (>1,000 ng/cartridge)
were observed to decrease accuracy of
identification for n-dodecane, n-propyl
acetate, di-A?-butyl ether, 2-methylbenzo-
furan,  pyridine, m-ethyltoluene, 1,3,5-
trimethylbenzene, naphthalene, 2-meth-
ylthiophene, and acetic acid.
  A comparison was made of the ability of
each interpretative procedure to  accu-
rately  identify the chemicals in each
mixture. The overall best percent correct
identification was achieved on data with-
out deconvolution (-D) using the INCOS
software (75.2%). The remaining proce-
dures ranked as follows: manual (70.7%)
>  RTI/MSSS (with deconvolution (+D),
60.9%) > INCOS (+D,58.2%)> RTI/MSSS
(-D, 59.0%) > MSFS (-D, 49.5%) > MSFS
(+D,  48.4%). In general, the use of a
deconvolution algorithm did not always
appear to  aid in  correctly  identifying
chemicals in these synthetic  mixtures.
  The automated interpretative methods
were compared to manual interpretation
for the percentage of correctly identified
chemicals in a mixture. The  highest
percent agreement (-D,  63.6%) was be-
tween INCOS and manual interpretation;
the lowest percent agreement was  be-
tween  manual interpretation and MSFS
(+D, 43.0%). When comparing only auto-
mated  procedures,  the  best agreement
(55.3%) was between INCOS (-D) and
RTI/MSSS (+D).
  Because environmental air samples
represented truly unknown mixtures, the
correct answers were unknown and, thus,
all comparative statistics on the answers
(identities) were made relative to manual
interpretation. The following observations
were apparent:

  1.  A higher percent agreement occur-
     red  between  manual and RTI/
     MSSS or INCOS methods with the
     higher-volume (thus, higher chem-
     ical levels) samples than with low-
     volume samples.

  2.  The application of deconvolution to
     the raw mass spectra  caused  the
     percent agreement to decrease for
     many samples.

  3.  The  highest overall  agreement
     (53.9%) across all samples was
     between manual interpretation and
     INCOS interpretation (+D).

  4.  INCOS more often agreed with the
     answers  obtained  manually than
     with the RTI/MSSS, when compar-
     ing by chemical classes.
 5.  Aldehydes and halogenated aroma-
     tics were always ranked in the top
     six best agreements regardless of
     automated method used.

 6.  Sulfur  compounds always ranked
     among the worst three in percent
     agreements regardless of the auto-
     mated method.

  For the chemical classes studied, this
program established that  accuracy can
best  be achieved by applying either the
INCOS or manual  interpretative proce-
dures. The accuracy is still unacceptably
low.  Several recommendations  were
offered:

 1.  Evaluate additional algorithms, e.g.,
     the STIRRS (Cornell University).

 2.  Include the use of relative retention
     data for chemicals analyzed under
     the same GC/MS conditions  as
     environmental samples and evalu-
     ate the change in accuracy, if any,
     by their inclusion.

 3.  Expand the list of chemical classes
     to  include polynuclear aromatics,
     pesticides, etc.

 4.  Attempt to optimize the Rindfleisch
     deconvolution algorithm.

 5.  After accomplishing items  1-4
     above, consider the need for further
     research on  development  of auto-
     mated algorithms or optimization of
     those appearing to be most promis-
     ing.

-------
     f. D. Pellizzari, T.  Hartwell, and J. Crowder are with the Research Triangle
       Institute, Research Triangle Park, NC 27709.
     Leonard Stockburger is the EPA Project Officer (see below).
     The complete report, entitled "A Comparative Evaluation of GC/MS Data Analysis
       Processing." (Order No. PB 85-125 664; Cost: $ 16.00. subject to change) will be
       available only from:
             National Technical Information Service
             5285 Port Royal Road
             Springfield, VA 22161
             Telephone: 703-487-4650
     The EPA Project Officer can be contacted at:
             Atmospheric Sciences Research Laboratory
             U.S. Environmental Protection Agency
             Research Triangle Park, NC 27711
United States
Environmental Protection
Agency
Center for Environmental Research
Information
Cincinnati OH 45268
     BULK RATE
POSTAGE & FEES PAID
        EPA
   PERMIT No. G-35
Official Business
Penalty for Private Use $300
                                                                                •e, U.S. GOVERNMENT PRINTING OFFICE: 1985-559-111/10752

-------