March 2006 Report
Revised-June 2007
Bibliometric Analysis of Papers on Topics
Related to Global Change
This is a revised bibliometric analysis of the papers prepared by intramural and extramural
researchers of the U.S. Environmental Protection Agency (EPA) on topics related to global
change (GC). This analysis was revised in June 2007 because the journals were initially
categorized into the fields used by Thomson Scientific's Essential Science Indicators (ESI) using
information provided in Thomson's Journal Citation Reports (JCR); for this revised analysis, the
journals were categorized into ESI fields using the journal category list by ESI that is available
on the Internet at http://in-cites.com/journal-list/index.html. The Journal List for ESI was made
available in 2006 and the current list contains all of the 12,734 journals covered for ESI up to
December 31, 2006. This list is updated bimonthly by Thomson. This revised bibliometric
analysis will allow comparison of the results of this 2005 analysis to those of the analysis
performed in 2007.
This is a bibliometric analysis of the papers prepared by intramural and extramural researchers of
on topics related to global change. For this analysis, 341 papers, reports, and books were
reviewed, and they were published from 1998 to 2005. These publications were cited 3,694 times
in the journals covered by Thomson's Web of Science1 Of these 341 publications, 293 (86%)
have been cited at least once in a journal.
Searches of Thomson's Web of Science, Scopus2, and Google were conducted to obtain times
cited data for the global change journal publications. The analysis was completed using
Thomson's ESI and JCR as benchmarks. ESI provides access to a unique and comprehensive
compilation of essential science performance statistics and science trends data derived from
Thomson's databases. For influence and impact measures, ESI employs both total citation counts
by field and cites per paper scores. The former reveals gross influence while the latter shows
weighted influence, also called impact. JCR is a recognized authority for evaluating journals. It
presents quantifiable statistical data that provide a systematic, objective way to evaluate the
world's leading journals and their impact and influence in the global research community. The
two key measures used in this analysis to assess the journals in which the EPA global change
papers are published are the Immediacy Index and the Impact Factor. The Immediacy Index is a
measure of how quickly the "average article" in a journal is cited. This index indicates how
often articles published in a journal are cited within the same year and it is useful in comparing
how quickly journals are cited. The Impact Factor is a measure of the frequency with which the
"average article" in a journal has been cited in a particular year. The Impact Factor helps
Thomson's Web of Science provides access to current and retrospective multidisciplinary information from
approximately 8,700 of the most prestigious, high impact research journals in the world. Web of Science also
provides cited reference searching.
2
Scopus is a large abstract and citation database of research literature and quality Web sources designed to
support the literature research process. Scopus offers access to 15,000 titles from 4,000 different publishers,
more than 12,850 academic journals (including coverage of 535 Open Access journals, 750 conference
proceedings, and 600 trade publications), 27 million abstracts, 245 million references, 200 million scientific
Web pages, and 13 million patent records.
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
evaluate a journal's relative importance, especially when compared to other journals in the same
field.
The report is divided into three sections. The first section presents an analysis of all 341 global
change papers analyzed by ESI field (e.g., Environment/Ecology, Geosciences, Plant & Animal
Science, Engineering). The second section presents an analysis of the global change papers by
year of publication. The third section presents an analysis of the global change papers by focus
area (e.g., air quality, regional assessment, ecosystems). In each section, a summary of the
results is provided as well as the data evaluated and a description of the analysis.
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
I. Analysis of Global Change Publications by ESI Field
SUMMARY OF RESULTS
l. About one-fifth of the global change publications are highly cited papers. A review of the
citations indicates that 67 (19.6%) of the global change papers qualify as highly cited when using
the ES/ criteria for the top 10% of highly cited publications. This is nearly twice the number
expected for a typical program. Eight (2.3%) of the global change papers qualify as highly cited
when using the criteria for the top 1 %, which is more than twice the number expected. One
(0.3%) of these papers qualifies as very highly cited (in the top 0.1%), which is 3 times the
number expected. It is not surprising that no papers actually meet the 0.01 % threshold for the
most highly cited papers because the expected number for this program is 0.03 papers in the
most highly cited category.
2. The global change papers are more highly cited than the average paper. Using the ESI
average citation rates for papers published by field as the benchmark, in 9 of the 14 fields in
which the EPA global change papers were published, the ratio of actual to expected cites is
greater than 1, indicating that the global change papers are more highly cited than the average
papers in those fields. For all 14 fields combined, the ratio of total number of cites to the total
number of expected cites (3,694 to 2103.25) is 1.8, indicating that the global change papers are
more highly cited than the average paper.
3. Nearly one-third of the global change papers are published in very high impact journals.
Ninety-nine (99) of the 341 papers were published in the top 10% of journals ranked by JCF?
Impact Factor, representing 29% of EPA's global change papers. This number is about 3 times
higher than expected. One-hundred seven (107) of the 341 papers appear in the top 10% of
journals ranked by JCf? Immediacy Factor, representing 31% of EPA's global change papers.
This number is more than 3 times higher than expected.
4. Eight of the global change papers qualify as hot papers. Using the hot paper thresholds
established by ESI as a benchmark, 8 hot papers, representing 2.4% of the global change
papers, were identified in the analysis. Hot papers are papers that were highly cited shortly after
they were published. The number of global change hot papers is 24 times higher than expected.
5. The authors of the global change papers cite themselves much less than the average
author. Two-hundred fifteen (215) of the 3,694 cites are author self-cites. This 5.8% author self-
citation rate is well below the accepted range of 10-30% author self-citation rate.
6. Twenty-seven of the authors of the global change papers are included in
ISIHighlyCited.com, which is a database of the world's most influential researchers who have
made key contributions to science and technology during the period from 1981 to 1999.
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
Highly Cited Global Change Publications
The 341 global change papers reviewed for this analysis covered 14 of the 22 ESI fields. The
distribution of the papers among these 14 fields and the number of citations by field are
presented in Table 1.
Table 1. Global Change Papers by ESI Fields
ESI Field
Environment/Ecology
Geosciences
Plant & Animal Science
Multidisciplinary
Clinical Medicine
Engineering
Microbiology
Biology & Biochemistry
Immunology
Social Sciences, general
Economics & Business
Agricultural Sciences
Physics
Computer Science
No. of
Citations
1,521
680
462
419
211
203
85
60
20
14
11
6
2
0
Total =
3,694
No. of EPA
GC Papers
148
54
48
7
19
41
4
8
2
5
2
1
1
1
Total =
341
Average
Cites/Paper
10.28
12.59
9.62
59.86
7.00
4.95
21.25
7.50
10.0
2.80
5.50
1.00
2.00
0.00
10.83
There are 67 (19.65% of the papers analyzed) highly cited EPA global change papers in 7 of the
14 fields—Environment/Ecology, Geosciences, Multidisciplinary, Plant & Animal Science,
Engineering, Clinical Medicine, and Economics & Business—when using the ESI criteria for the
top 10% of papers. Table 2 shows the number of EPA papers in those 7 fields that meet the top
10% threshold in ESI Eight (2.35%) of the papers analyzed qualify as highly cited when using
the ESI criteria for the top 1% of papers. These papers cover five fields—Multidisciplinary,
Environment/Ecology, Geosciences, Plant & Animal Science, and Engineering. Table 3 shows
the 8 papers by field that meet the top 1% threshold in ESI The citations for these 10 papers
are provided in Tables 4 through 8. There was 1(0.3%) very highly cited global change paper in
the Multidisciplinary field. This paper, which met the top 0.1% threshold in ESI, is listed in
Table 9. None of the global change papers actually met the top 0.01% threshold in ESI, which
is to be expected.
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
Table 2. Number of Highly Cited Global Change Papers by Field (top 10%)
ESI Field
Environment/Ecology
Geosciences
Multidisciplinary
Plant & Animal Science
Engineering
Clinical Medicine
Economics & Business
Citations
773
485
405
262
128
126
10
Total =
2,189
No. of
Papers
27
13
5
11
7
3
1
Total =
67
Average
Cites/Paper
28.63
37.31
81.00
23.82
18.26
42.00
10.00
32.67
% of EPA
Papers in
Field
18.24%
24.07%
71.43%
22.92%
17.07%
15.79%
50.00%
19.65%
Table 3. Number of Highly Cited Global Change Papers by Field (top 1%)
ESI Field
Multidisciplinary
Environment/Ecology
Geosciences
Plant & Animal Science
Engineering
Citations
304
215
134
79
52
Total =
784
No. of
Papers
2
2
2
1
1
Total = 8
Average
Cites/Paper
152.0
107.5
67.00
79.00
52.00
98.0
% of EPA
Papers in
Field
28.57%
1.35%
3.70%
2.08%
2.44%
2.35%
Table 4. Highly Cited Global Change Papers in the Field of Multidisciplinary (top 1%)
No. of
Cites
236
68
First Author
Root TL
Pascual M
Paper
Fingerprints of global warming on wild animals and
2003;421(6918):57-60.
Cholera dynamics and El Nino-Southern Oscillation
2000;289(5485): 1766-1769.
plants. Nature
Science
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
Table 5. Highly Cited Global Change Papers in the Field of
Environment/Ecology (top 1%)
No. of
Cites
141
74
First Author
National
Assessment
Team
Marsh DM
Paper
Climate Change Impacts on the United States
of Climate Variability and Change (Overview
2001.
The Potential Consequences
and Foundation reports),
Metapopulation dynamics and amphibian conservation. Conservation
fl/o/ogy2001;15(l):40-49.
Table 6. Highly Cited Global Change Papers in the Field of Geosciences (top 1%)
No. of
Cites
61
73
First Author
Pielke RA
Chase TN
Paper
Influence of the spatial distribution of vegetation and soils on the
prediction of cumulus convective rainfall. Reviews of Geophysics
2001;39(2):151-177.
Simulated impacts of historical land cover changes on global climate in
northern winter. Climate Dynamics 2000;16(2-3):93-105.
Table 7. Highly Cited Global Change Paper in the Field of
Plant & Animal Science (top 1%)
No. of
Cites
79
First Author
Moran MA
Paper
Carbon loss and optical property changes during long-term
photochemical and biological degradation of estuarine dissolved organic
matter. Limnology and Oceanography 2000;45(6): 1254-1264.
Table 8. Highly Cited Global Change Papers in the Field of Engineering (top 1%)
No. of
Cites
52
First Author
Douglas EM
Paper
Trends in floods and low flows in the United States: impact of spatial
correlation. Journal of Hydrology 2000;240(1-2):90-105.
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
Table 9. Very Highly Cited Global Change Paper in the
Field of Multidisciplinary (top 0.1%)
No. of
Cites
236
First Author
Root TL
Paper
Fingerprints of global warming on wild animals and plants.
2003;421(6918):57-60.
Nature
Ratio of Actual Cites to Expected Citation Rates
The expected citation rate is the average number of cites that a paper published in the same
journal in the same year and of the same document type (article, review, editorial, etc.) has
received from the year of publication to the present. Using the ESI average citation rates for
papers published by field as the benchmark, in 9 of the 14 fields in which the EPA global change
papers were published, the ratio of actual to expected cites is greater than 1, indicating that the
global change papers are more highly cited than the average papers in those fields (see Table 10).
Table 10. Ratio of Actual Cites to Expected Cites for Global Change Papers by Field
ESI Field
Agricultural Sciences
Biology & Biochemistry
Clinical Medicine
Computer Science
Economics & Business
Engineering
Environment/Ecology
Geosciences
Immunology
Microbiology
Multidisciplinary
Physics
Plant & Animal Science
Social Sciences, general
Total Cites
6
60
211
0
11
203
1,521
680
20
85
419
2
462
14
Expected Cite
Rate
6.98
64.12
148.50
0.63
2.36
112.83
1,041.40
354.04
28.94
53.17
28.97
3.96
247.31
10.07
Ratio
0.86
0.94
1.42
0.00
4.66
1.80
1.46
1.92
0.69
1.60
14.46
0.50
1.87
1.39
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
JCR Benchmarks
Impact Factor. The JCR Impact Factor is a well known metric in citation analysis. It is a
measure of the frequency with which the "average article" in a journal has been cited in a
particular year. The Impact Factor helps evaluate a journal's relative importance, especially
when compared to others in the same field. The Impact Factor is calculated by dividing the
number of citations in the current year to articles published in the 2 previous years by the total
number of articles published in the 2 previous years.
Table 11 indicates the number of global change papers published in the top 10% of journals,
based on the JCR Impact Factor. Ninety-nine (99) of 341 papers were published in the top 10%
of journals, representing 29.0% of EPA's global change papers. This indicates that nearly one-
third of the global change papers are published in the highest quality journals as determined by
the JCR impact factor, which is nearly 3 times the expected percentage.
Table 11. Global Change Papers in Top 10% of Journals by JCR Impact Factor
EPA Global
Change
Papers in
that Journal
10
9
7
7
6
6
5
5
5
4
4
4
3
2
2
2
2
Journal
Environmental Health Perspectives
Journal of Geophysical Research- Atmospheres
Ecological Applications
Ecosystems
Global Change Biology
Journal of Climate
Epidemiology
Conservation Biology
Environmental Science & Technology
Lancet
Proceedings of the National Academy of Sciences of the
United States of America
Limnology and Oceanography
Applied and Environmental Microbiology
Nature
Ecology
Plant Cell and Environment
New Phytologist
Impact
Factor
(IF)
3.929
2.839
3.287
3.241
4.333
3.500
3.840
3.672
3.557
21.713
10.452
3.024
3.810
32.182
4.104
3.634
3.355
JCR IF
Rank
439
831
623
642
358
558
459
504
540
20
88
737
470
9
394
517
603
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
EPA Global
Change
Papers in
that Journal
2
2
1
1
1
1
1
1
1
1
1
1
1
1
Total = 99
Journal
American Journal of Public Health
Bioscience
Science
JAMA- Journal of the American Medical Association
Reviews of Geophysics
British Medical Journal
Emerging Infectious Diseases
Trends in Parasitology
Journal of Infectious Diseases
American Journal of Epidemiology
Microbes and Infection
Climate Dynamics
Oecologia
Agricultural and Forest Meteorology
Impact
Factor
(IF)
3.241
3.041
31.853
24.831
8.667
7.038
5.643
5.497
4.943
4.933
3.753
3.497
2.899
2.811
JCR IF
Rank
642
730
10
15
114
169
230
239
287
290
480
561
800
844
Immediacy Index. The JCR Immediacy Index is a measure of how quickly the average article in
a journal is cited. It indicates how often articles published in a journal are cited within the year
they are published. The Immediacy Index is calculated by dividing the number of citations to
articles published in a given year by the number of articles published in that year.
Table 12 indicates the number of global change papers published in the top 10% of journals,
based on the JCR Immediacy Index. One-hundred seven (107) of the 341 papers appear in the
top 10% of journals, representing 31.4% of the global change papers. This indicates that
approximately one-third of the global change papers are published in the highest quality journals
as determined by the JCR immediacy index, which is more than 3 times higher than the expected
percentage.
Table 12. Global Change Papers in Top 10% of Journals by JCR Immediacy Index
EPA Papers in
that Journal
17
Journal
Climatic Change
Immediacy
Index
(II)
1.235
JCR II
Rank
195
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
EPA Papers in
that Journal
10
9
7
7
6
5
5
5
4
4
3
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
Journal
Environmental Health Perspectives
Journal of Geophysical Research- Atmospheres
Ecosystems
Ecological Applications
Journal of Climate
Epidemiology
Conservation Biology
Environmental Science & Technology
Lancet
Proceedings of the National Academy of Sciences of the
United States of America
Journal of Biogeography
Nature
New Phytologist
Bioscience
American Journal of Public Health
Plant Cell and Environment
Ecology
Science
JAMA- Journal of the American Medical Association
British Medical Journal
Reviews of Geophysics
Ambio
Emerging Infectious Diseases
Journal of Infectious Diseases
American Journal of Epidemiology
Trends in Parasitology
Aquatic Sciences
Hydrobiologia
Immediacy
Index
(II)
1.202
0.617
2.048
0.747
0.528
0.864
0.744
0.623
5.017
1.923
0.514
6.089
0.876
0.863
0.723
0.605
0.590
7.379
5.499
3.039
1.714
1.435
1.350
1.105
0.842
0.819
0.800
0.681
JCRll
Rank
202
630
76
466
795
354
468
617
12
89
827
5
349
356
489
653
676
3
9
35
110
156
169
229
373
393
413
532
10
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
EPA Papers in
that Journal
1
1
Total = 107
Journal
Tellus Series B-Chemical and Physical Meteorology
Theoretical and Applied Climatology
Immediacy
Index
(II)
0.610
0.564
JCRll
Rank
646
720
Hot Papers
ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers
in different fields, but the time frame for citing and cited papers is much shorter—papers must be
cited within 2 years of publication and the citations must occur in a 2-month time period. Papers
are assigned to 2-month periods and thresholds are set for each period and field to select 0.1% of
papers. There were no hot papers identified for the current 2-month period (i.e., January-
February 2006), but there were a number of hot papers identified from previous periods.
Using the hot paper thresholds established by ESI as a benchmark, 8 hot papers, representing
2.4% of the global change papers, were identified in five fields—Environment/Ecology,
Engineering, Multidisciplinary, Plant & Animal Science, and Geosciences. The hot papers are
listed in Table 13.
Table 13. Hot Papers Identified Using ESI Thresholds
Field
Environment/
Ecology
ESI Hot
Papers
Threshold
4
4
4
4
No. of Cites
in 2-Month
Period
8 cites in
May 2000
5 cites in
May 2000
5 cites in
May 2000
4 cites in
October-
November
2001
Paper
Polsky C, et al. The Mid-Atlantic Region and its
climate: past, present, and future. Climate
Research 2000;14(3):161-173.
Rose A, et al. Simulating the economic impacts
of climate change in the Mid-Atlantic Region.
Climate Research 2000;14(3): 175-183.
Fisher A, et al. The Mid-Atlantic Regional
Assessment: motivation and approach. Climate
Research 2000; 14(3): 153-159.
Patz JA, et al. The potential health impacts of
climate variability and change for the United
States: executive summary of the health sector of
the US National Assessment. Environmental
Health Perspectives 2000;108(4):367-376.
11
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
Field
Engineering
Multidisciplinary
Plant & Animal
Science
Geosciences
ESI Hot
Papers
Threshold
5
9
4
4
No. of Cites
in 2-Month
Period
7 cites in
June -July
2002
9 cites in
September-
October
2003
5 cites in
December
2002
4 cites in
October-
November
2001
Paper
Douglas EM, et al. Trends in floods and low
flows in the United States: impact of spatial
correlation. Journal of Hydrology 2000;240(1-
2):90-105.
Root TL, et al. Fingerprints of global warming on
wild animals and plants. Nature
2003;421(6918):57-60.
Sousounis PJ, Grover EK. Potential future
weather patterns over the Great Lakes region.
Journal of Great Lakes Research 2002;28(4):496-
520.
Hogrefe C, et al. Simulating regional-scale ozone
climatology over the eastern United States: model
evaluation. Atmospheric Environment
2004;38(17):2627-2638.
Author Self-Citation
Self-citations are journal article references to articles from that same author (i.e., the first author).
Because higher author self-citation rates can inflate the number of citations, the author self-
citation rate was calculated for the global change papers. Of the 3,694 total cites, 215 are author
self-cites—a 5.8% author self-citation rate. Garfield and Sher3 found that authors working in
research-based disciplines tend to cite themselves on the average of 20% of the time.
MacRoberts and MacRoberts4 claim that approximately 10% to 30% of all the citations listed fall
into the category of author self-citation. Kovacic and Misak5 recently reported a 20% author self-
citation rate for medical literature. Therefore, the 5.8% self-cite rate for the global change papers
is well below the range for author self-citation.
Highly Cited Researchers
A search of Thomson's ISIHighlyCited. com revealed that 27 (3.5%) of the 773 authors of the
global change papers are highly cited researchers. ISIHighlyCited.com is a database of the
world's most influential researchers who have made key contributions to science and technology
Garfield E, Sher IH. New factors in the evaluation of scientific literature through citation indexing. American
Documentation 1963;18(July):195-210.
4 MacRoberts MH, MacRoberts BR. Problems of citation analysis: a critical review. Journal of the American
Society of Information Science 1989;40(5):342-349.
5 Kavaci N, Misak A. Author self-citation in medical literature. Canadian Medical Association Journal
2004;170(13):1929-1930.
12
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
during the period from 1981 to 1999. The highly cited researchers identified during this analysis
of the global change publications are presented in Table 14.
Table 14. Highly Cited Researchers Authoring Global Change Publications
Highly Cited
Researcher
Ankley, Gerald T
Brown, Sandra L
Caldwell, Martyn M
Callaghan, Terry V
Chase, Thomas N
Colwell, Rita R
Ehleringer, James
Elliott, Edward T
Giorgi, Filippo
Goldberg, Richard
Jacob, Daniel J
Lauenroth, William K
Lugo, Ariel E
Ojima, Dennis S
Pacala, Stephen
Palmer, T.N.
Parton, William J
Pielke, Roger A
Rind, David H
Running, Steven W
Sala, Osvaldo E
Schimel, David S
Schwartz, Joel D
Shugart, Herman H
Teramura, Alan H
Zeger, Scott L
Zepp, Richard G
Total = 27
Affiliation ESI Field
U.S. EPA
Winrock International
Utah State University
University of Sheffield
NINDS
Canon U.S. Life Sciences
University of Utah
University of Nebraska
Abdus Salam International Centre
for Theoretical Physics
Columbia University
Harvard University
Colorado State University
USDA
Colorado State University
Princeton University
European Center for Medium Range
Weather Forecasts
Colorado State University
Colorado State University
NASA Goddard
University of Montana
Brown University
National Center for Atmospheric
Research
Harvard University
University of Virginia
University of Hawaii
Johns Hopkins University
U.S. EPA
Environment/Ecology
Environment/Ecology
Environment/Ecology
Environment/Ecology
Neuroscience
Microbiology
Environment/Ecology
Environment/Ecology
Geosciences
Plant & Animal Science
Geosciences
Environment/Ecology
Environment/Ecology
Environment/Ecology
Ecology/Environment
Geosciences
Environment/Ecology
Geosciences
Geosciences
Environment/Ecology
Environment/Ecology
Environment/Ecology
Environment/Ecology
Environment/Ecology
Plant & Animal Science
Mathematics
Environment/Ecology
13
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
II. Analysis of Global Change Publications by Year of Publication
This section of the report presents an analysis of the global change papers by year of publication
from 1998 to 2005. The data are presented by year in Table 15, which includes eight key
bibliometric parameters.
SUMMARY OF RESULTS
1. About one-fifth of the global change publications are highly cited papers (this is twice the number
expected for a typical program). From 1998 to 2005, the percentage of global change papers that
qualified as highly cited when using the ES/ criteria for the top 10% of highly cited publications ranges
from a low of 7.4% to a high of 30.9%. From 2000 to 2003, there were 8 papers that qualified as very
highly cited when using the ES/ criteria for the top 1 %, with the percentages ranging from 2.4% to 7.7%.
These percentages are, on average, about 2.6 times higher than would be expected for a typical
program. One paper published in 2003 meets the ESI criteria for the top 0.1 %, which is about 3 times
higher than would be expected for a typical program. None of the global change papers meet the ESI
criteria for the most highly cited papers (top 0.01%). The expected number of papers in this top category
for a program of this size would be 0.03 papers, so the absence of papers in this top category is not
surprising.
2. The global papers are more highly cited than the average paper. Using the ESI average citation
rates for papers published by field as the benchmark, the ratio of actual to expected cites is greater than
1 for all but one of the years analyzed (i.e.,2004). This indicates that the global change papers are more
highly cited than the average papers published in these fields.
3. Nearly one-third of the global change papers are published in high impact journals as determined
by the JCF? Impact Factor of the journals in which the papers are published. For every year analyzed the
number of global change papers published in high impact journals (the top 10% of journals) exceeds the
expected 10% as determined by the JCf? Impact Factor and Immediacy Index of the journals. The
percentage of papers in high impact journals (by Impact Factor) ranges from 14.3% to 45.2%, which is
1.4 to 4.5 times higher than expected. The percentage of papers in high impact journals (by Immediacy
Index) ranges from 14.3% to 38.1%, which is 1.4 to 3.8 times higher than expected.
4. The percentage of publications cited one or more times is consistently high from 1998 to 2003.
The number declines as expected in the more recent years (i.e., 2004 and 2005) because most
publications are not cited until a year or longer after they are published.
5. The authors of the global change papers cite themselves less than the average self-citation rate.
The author self-citation rate for most years is well below the accepted range of 10-30% author self-
citation rate. The only exception is in 2005, which is expected because most publications are not cited by
other researchers until a year or longer after they are published.
6. There were hot papers published in 2000,2002,2003, and 2004. The absence of hot papers in 2005
is not surprising given the fact that the period during which the citations analyzed for determining hot
paper status was incomplete.
14
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
The results of the analysis are presented below and the numbers link the findings with the
corresponding data in Table 15.
1. No. of Global Change Papers Analyzed—The number of global change publications has
ranged from 27 to 56 per year over the period analyzed (i.e., 1998 to 2005).
2. Total No. of Highly Cited Publications—ESI identifies four thresholds of highly cited
papers—those in the top 10%, 1%, 0.1%, and 0.01%. It is extraordinary for a publication to
meet the threshold for the top 0.01%; these publications are rare and should not be expected
in every program. Using the ESI thresholds for the top 10% cited papers, about 20% of the
global change publications are highly cited papers (this is twice the number expected for a
typical program). From 1998 to 2005, the percentage of global change papers that qualify as
highly cited when using the ESI criteria for the top 10% of highly cited publications ranges
from a low of 7.4% to a high of 30.9%. From 2000 to 2003, there were 8 papers that qualify
as very highly cited when using the ESI criteria for the top 1%, with the percentage ranging
from 0% to 7.7%. These percentages are, on average, about 3,5 times higher than would be
expected for a typical program. One paper published in 2003 meets the ESI criteria for the
top 0.1%, which is about 3 times higher than would be expected for a typical program. None
of the global change papers meet the ESI criteria for the most highly cited papers (top
0.01%). The expected number of papers in this top category for a typical program of this size
would be 0.03 papers, so the absence of papers in this top category is not surprising.
3. Ratio of Actual to Expected Cites—The global papers are more highly cited than the
average paper. Using the ESI average citation rates for papers published by field as the
benchmark, the ratio of actual to expected cites is greater than 1 for all but two of the years
analyzed (i.e., 2004 and 2005). This indicates that the global change papers are more highly
cited than the average papers published in these fields.
4. No. of Papers in High Impact Journals by Impact Factor—Nearly one-third of the global
change papers are published in high impact journals as determined by the JCR Impact Factor
of the journals in which the papers are published. For every year analyzed, the number of
global change papers published in high impact journals (the top 10% of journals) exceeds the
expected 10% as determined by the JCR Impact Factor of the journals. The percentage of
papers in high impact journals (by Impact Factor) ranges from 14.3% to 45.2%, which is 1.4
to 4.5 times higher than expected.
5. No. of Papers in High Impact Journals by Immediacy Index—Nearly one-third of the
global change papers are published in high impact journals as determined by the JCR
Immediacy Index of the journals in which the papers are published. For every year analyzed,
the number of global change papers published in high impact journals (the top 10% of
journals) exceeds the expected 10% as determined by the JCR Immediacy Index of the
journals. The percentage of papers in high impact journals (by Immediacy Index) ranges
from 14.3% to 38.1%, which is 1.4 to 3.8 times higher than expected.
6. Total No. of Publications Cited One or More Times—The percentage of publications cited
one or more times is consistently high (88% or greater) from 1998 to 2003. The number
declines as expected in the more recent years (i.e., 2004 and 2005) because most publications
are not cited until a year or longer after they are published.
15
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
7. Total No. of Author Self Cites—The authors of the global change papers cite themselves
less than the average self-citation rate. The author self-citation rates in Table 15 are well
below the accepted range of 10-30% author self-citation rate. The only exception is in 2005,
which is expected because most publications are not cited by other researchers until a year or
longer after they are published.
8. No. of Hot Papers—Using the hot paper thresholds established by ESI as a benchmark, there
were hot papers published in 2000, 2002, 2003, and 2004. Because only the top 0.1% of
papers are selected as hot papers for each ESI field, the percentages of global change hot
papers identified in Table 15 are 18 to 96 times higher than expected for these years. The
percentage of hot papers in 2004 and 2005 is probably low because this analysis was
conducted before the 2 year period in which hot papers must be cited was completed. ESI
establishes citation thresholds for hot papers, which are selected from the highly cited papers
in different fields, but the time frame for citing and cited papers is much shorter—papers
must be cited within 2 years of publication and the citations must occur in a 2-month time
period. Papers are assigned to 2-month periods and thresholds are set for each period and
field to select 0.1% of papers. For papers published in 2004, the 2-year period of
consideration would not conclude until 2006. Similarly, for papers published in 2005, the 2-
year period of consideration would not conclude until 2007. The original bibliometric
analysis was completed in March 2006; therefore, the number of hot papers among those
published in 2004 and 2005 would be expected to be lower than if the analysis was
completed in January 2008.
Table 15. Key Bibliometric Parameters for Global Change Papers By Year (1998 to 2005)
ANALYSIS PARAMETERS
1. No. of Global Change Papers
Analyzed
2. No. of Highly Cited
Publications That Met the
Top 10% Threshold
(Percentage)
No. of Highly Cited
Publications That Met the
Top 1% Threshold
(Percentage)
No. of Highly Cited
Publications That Met the
Top 0.1% Threshold
(Percentage)
No. of Highly Cited
Publications That Met the
Top 0.01% Threshold
(Percentage)
1998
35
9
(25.7%)
0
(0%)
0
(0%)
0
(0%)
1999
42
9
(21.4%)
0
(0%)
0
(0%)
0
(0%)
2000
52
9
(17.3%)
4
(7.7%)
0
(0%)
0
(0%)
2001
42
13
(30.9%)
3
(7.1%)
0
(0%)
0
(0%)
2002
56
12
(21.4%)
0
(0%)
0
(0%)
0
(0%)
2003
41
9
(21.9%)
1
(2.4%)
1
(2.4%)
0
(0%)
2004
46
4
(8.7%)
0
(0%)
0
(0%)
0
(0%)
2005
27
2
(7.4%)
0
(0%)
0
(0%)
0
(0%)
16
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
ANALYSIS PARAMETERS
3. Expected No. of Citations
Calculated Using the Average
Citation Rate
Total No. of Times Cited for
All Publications
Ratio of Actual Cites to
Expected Cites
4. No. of Papers in High Impact
Journals by Impact Factor
(Percentage)
5. No. of Papers in High Impact
Journals by Immediacy Index
(Percentage)
6. No. of Publications Cited One
or More Times
(Percentage)
7. Total No. of Author Self
Cites (Percentage)
8. No. of Hot Papers
(Percentage)
1998
376.59
630
1.67
5
(14.3%)
6
(17.1%)
34
(97.1%)
61
(9.7%)
0
(0%)
1999
382.38
528
1.38
7
(16.7%)
6
(14.3%)
40
(95.2%)
49
(9.3%)
0
(0%)
2000
489.43
817
1.67
17
(32.7%)
14
(26.9%)
51
(98.1%)
35
(4.3%)
5
(9.6%)
2001
328.86
791
2.41
19
(45.2%)
16
(38.1%)
41
(97.6%)
26
(3.3%)
0
(0%)
2002
295.71
434
1.47
20
(35.7%)
19
(33.9%)
54
(96.4%)
25
(5.8%)
1
(1.8%)
2003
144.18
419
2.91
8
(19.5%)
13
(31.7%)
36
(87.8%)
10
(2.4%)
1
(2.4%)
2004
78.10
67
0.86
14
(30.4%)
15
(32.6%)
31
(67.4%)
5
(7.5%)
1
(2.2%)
2005
8.00
8
1.00
9
(33.3%)
9
(33.3%)
6
(22.2%)
4
(50.0%)
0
(0%)
III. Analysis of Global Change Publications by Focus Area
This section of the report presents an analysis of the global change papers by focus area (i.e., air
quality, ecosystems, human health, regional assessment, and water quality). The data are
presented by focus area in Table 16, which includes eight key bibliometric parameters.
17
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
SUMMARY OF RESULTS
1. About one-fifth of the global change publications are highly cited papers (this is twice the
number expected for a typical program). The percentage of global change papers that qualify as
highly cited when using the ES/ criteria for the top 10% of highly cited publications ranges from 2.4%
for the air quality papers to 35.0% for the water quality papers. The water quality and human health
focus areas have the highest percentages of highly cited publications when using the ES/ criteria for
the top 10%, and the number of highly cited papers in these areas is 3.5 to 2.9 times higher than
expected. The ecosystems and water quality focus areas hold the lead positions when using the ESI
criteria for the top 1 % of papers, and the number of very highly cited papers in these areas is 2.9 to
2.5 times higher than expected. Only one paper in the ecosystems focus area meets the ESI criteria
for the top 0.1 % of papers, which is 6 times higher than the expected number for this focus area. None
of the global change papers meet the ESI criteria for the most highly cited papers (top 0.01 %), which is
expected.
!. The global papers are more highly cited than the average paper. Using the ESI average citation
rates for papers published by field as the benchmark, the ratio of actual to expected cites is greater
than 1 for all but the air quality focus area. This indicates that the global change papers are more
highly cited than the average papers published in these fields.
3. Nearly one-third of the global change papers are published in high impact journals as
determined by the JCR Impact Factor and Immediacy Index of the journals in which the papers are
published. For every year analyzed, the number of global change papers published in high impact
journals (the top 10% of journals) exceeds the expected 10% as determined by the JCR Impact Factor
and Immediacy Index of the journals. The percentage of papers in high impact journals (by Impact
Factor) for air quality, ecosystems, and human health ranges from 21.4% to 75.6%, which is 2 to 8
times higher than expected. The percentage of water quality papers published in high impact journals
(by Impact Factor) is the same as that expected (i.e., 10%). None of the regional assessment papers
are published in high impact journals, but this could be attributed to the fact that this focus area had
more non-journal publications than the other focus areas. The percentage of papers in high impact
journals (by Immediacy Index) for all five focus areas is higher than expected, ranging from 11.4% to
65.8%, which is 1.1 to 7 times higher than expected.
4. In four of the five focus areas (i.e., ecosystems, human health, regional assessment, and water
quality) the percentage of publications cited one or more times is very high (i.e., 88% to 95%).
For the air quality focus area the percentage drops to approximately 60%, but this is attributed to high
percentage of air quality papers published after 2003.
5. The authors of the global change papers cite themselves less than the average self-citation
rate. The author self-citation rates range from 3.4% to 23.4%. The rates for the ecosystems, human
health, regional assessment, and water quality focus areas are well below the accepted range of 10-
30% author self-citation rate, and the rate for the air quality papers is 23.4%, which is within the
accepted range.
6. There were hot papers published in each of the five focus areas. The highest percentage of hot
papers (i.e., 9.1%) is in the regional assessment focus area, and the percentage of hot papers in the
three of the other focus areas is consistent (i.e., approximately 2.4%). These percentages are 24 to 91
times higher than expected.
18
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
The results of the analysis are presented below and the numbers link the findings with the
corresponding data in Table 16.
1. No. of Global Change Papers Analyzed—More than half of the global change publications
fall under the ecosystems focus area. The number of global change publications for the
remaining focus areas is approximately the same.
2. Total No. of Highly Cited Publications—ESI identifies four thresholds of highly cited
papers—those in the top 10%, top 1%, top 0.1%, and top 0.0.1%. It is extraordinary for a
publication to meet the threshold for the top 0.01%; these publications are rare and should
not be expected in every program. Using the £57 thresholds, about 20% of the global change
publications are highly cited papers (this is 2 times the number expected for a typical
program). The percentage of global change papers that qualify as highly cited when using
the ESI criteria for the top 10% of highly cited publications ranges from 2.4% for the air
quality papers to 35.0% for the water quality papers. The water quality and human health
focus areas have the highest percentages of highly cited publications when using the ESI
criteria for the top 10%, and the number of highly cited papers in these areas is 3.5 and 2.9
times higher than expected. The ecosystems and water quality focus areas hold the lead
positions when using the ESI criteria for the top 1% of papers, and the number of very highly
cited papers in these areas is 2.9 and 2.5 times higher than expected. Only one paper in the
ecosystems focus area meets the ESI criteria for the top 0.1% of papers, which is 6 times
higher than the expected number for this area. None of the global change papers meet the ESI
criteria for the most highly cited papers (top 0.01%). The expected number of papers in this
top category for a typical program of this size would be 0.03 papers, so the absence of papers
in this category is not surprising.
3. Ratio of Actual to Expected Cites—The global papers are more highly cited than the
average paper. Using the ESI average citation rates for papers published by field as the
benchmark, the ratio of actual to expected cites is greater than 1 for all but one of the focus
areas (i.e., air quality). This indicates that the global change papers are more highly cited
than the average papers published in these fields.
4. No. of Papers in High Impact Journals by Impact Factor—Nearly one-third of the global
change papers are published in high impact journals as determined by the Impact Factor of
the journals in which the papers are published. The percentage of papers in high impact
journals for air quality, ecosystems, and human health ranges from 21.4% to 75.6%, which is
2 to 8 times higher than expected for these three focus areas. The percentage of water quality
papers published in high impact journals is the same as that expected (i.e., 10%). None of
the regional assessment papers are published in high impact journals, but this could be
attributed to the fact that this focus area had more non-journal publications than the other
focus areas.
5. No. of Papers in High Impact Journals by Immediacy Index—Nearly one-third of the
global change papers are published in high impact journals as determined by the Immediacy
Index of the journals in which the papers are published. The percentage of papers published
in high impact journals ranges from 11.4% to 65.8%, which is 1.1 to 7 times higher than
expected for these five focus areas.
19
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
6. Total No. of Publications Cited One or More Times—In four of the five focus areas (i.e.,
ecosystems, human health, regional assessment, and water quality) the percentage of
publications cited one or more times is very high (i.e., 88% to 95%). For the air quality focus
area, the percentage drops to approximately 60%, but this can be attributed to the fact that
74% of the air quality papers are published after 2003 compared to 4% of the regional
assessment papers. In fact, 20% of the air quality papers were published in 2005, and only 6
of the 27 (22%) global change papers published in 2005 (all focus areas combined) have
been cited one or more times.
7. Total No. of Author Self Cites—The authors of the global change papers cite themselves
less than the average self-citation rate. The author self-citation rates range from 3.4% to
23.4%. The rates for the ecosystems, human health, regional assessment, and water quality
focus areas are well below the accepted range of 10-30% author self-citation rate, and the
rate for the air quality papers is 23.4%, which is within the accepted range.
8. No. of Hot Papers—Using the hot paper thresholds established by ESI as a benchmark, there
were hot papers published in each of the five focus areas. The highest percentage of hot
papers (i.e., 9.1%) is in the regional assessment focus area, and the percentage of hot papers
in three of the other focus areas is consistent (i.e., approximately 2.4%). Because only the
top 0.1% of papers are selected as hot papers for each ESI field, the percentages of global
change hot papers identified in Table 16 are 24 to 91 times higher than expected.
Table 16. Key Bibliometric Parameters for Global Change Papers by Focus Area
ANALYSIS PARAMETERS
1. No. of Global Change Papers
Analyzed
2. No. of Highly Cited
Publications That Met the Top
10% Threshold (Percentage)
No. of Highly Cited
Publications That Met the Top
1% Threshold (Percentage)
No. of Highly Cited
Publications That Met the Top
0.1% Threshold (Percentage)
No. of Highly Cited
Publications That Met the Top
0.01% Threshold (Percentage)
Focus Areas
Air Quality
42
1 (2.4%)
0 (0%)
0 (0%)
0 (0%)
Ecosystems
174
35(20.1%)
5 (2.9%)
1 (0.6%)
0 (0%)
Human
Health
41
12 (29.3%)
1 (2.4%)
0 (0%)
0 (0%)
Regional
Assessment
44
5(11.4%)
1 (2.3%)
0 (0%)
0 (0%)
Water
Quality
40
14 (35.0%)
1 (2.5%)
0 (0%)
0 (0%)
20
-------
Bibliometric Analysis of Papers on Topics Related to Global Change
ANALYSIS PARAMETERS
3. Expected No. of Citations
Calculated Using the Average
Citation Rate
Total No. of Times Cited for
All Publications
Ratio of Actual Cites to
Expected Cites
4. No. of Papers in High Impact
Journals by Impact Factor
(Percentage)
5. No. of Papers in High Impact
Journals by Immediacy Index
(Percentage)
6. No. of Publications Cited One
or More Times (Percentage)
7. Total No. of Author Self Cites
(Percentage)
8. No. of Hot Papers
(Percentage)
Focus Areas
Air Quality
94.18
64
0.68
9(21.4%)
1 1 (26.2%)
25 (59.5%)
15 (23.4%)
1 (2.4%)
Ecosystems
1,082.61
2,034
1.88
55(31.6%)
58 (33.3%)
154 (88.5%)
115(5.6%)
1 (0.57%)
Human
Health
332.93
655
1.97
31 (75.6%)
27 (65.8%)
36 (87.8%)
29 (4.4%)
1 (2.4%)
Regional
Assessment
284.56
356
1.25
0 (0%)
5(11.4%)
42 (95.4%)
12 (3.4%)
4(9.1%)
Water
Quality
308.97
585
1.89
4(10.0%)
6(15.0%)
36 (90.0%)
44 (7.5%)
1 (2.5%)
This bibliometric analysis was prepared by
Beverly Campbell of The Scientific Consulting Group
in Gaithersburg, Maryland
under EPA Contract No. EP-C-05-015
21
------- |