August 2007 Bibliometric Analysis for the U.S. Environmental Protection Agency/Office of Research and Development's Human Health Risk Assessment Research Program This is a bibliometric analysis of the papers prepared by researchers of the U.S. Environmental Protection Agency (EPA) for the Human Health Risk Assessment (HHRA) Research Program. For this analysis, a total of 420 publications published from 2001 to 2007 were reviewed. The 420 publications included 292 journal articles and 128 non-journal publications (e.g., technical reports, handbooks, books/book chapters). These publications were cited 2,152 times in the journals covered by Thomson's Web of Science1 and Scopus2. The 292 journal publications were cited 1,897 times in the journals and the 128 non-journal publications were cited 255 times in the journals. Of the 292 journal publications, 226 (77%) have been cited at least once in a journal. Of the 128 non-journal publications, 39 (30.5%) were cited at least once in a journal. Searches of Thomson Scientific's Web of Science and Scopus were conducted to obtain times cited data for the HHRA journal publications. Searches of Web of Science were used to obtain times cited data for the HHRA non-journal publications. The process for searching non-journal publications differed from that used for the journal publications in that the Cited Reference Search feature was used to identify the number of times the non-journal publications were cited in the journals covered by Web of Science. Such searching involves an iterative process to identify the citing references; for example, searches for EPA, Environmental Protection Agency, USEPA, the document number, the publisher, and so on, are conducted. The results of these different searches then are combined to eliminate duplicate citations and determine total times cited data. The times cited data for the non-journal publications in this report do not include the citations of these documents in other non-journal publications. The analysis was completed using Thomson's Essential Science Indicators (ESI) and Journal Citation Reports (JCR) as benchmarks. ESI provides access to a unique and comprehensive compilation of essential science performance statistics and science trends data derived from Thomson's databases. For this analysis, the ESI highly cited papers thresholds as well as the hot papers thresholds were used to assess the influence and impact of the HHRA publications. JCR is a recognized authority for evaluating journals. It presents quantifiable statistical data that provide a systematic, objective way to evaluate the world's leading journals and their impact and influence in the global research community. The two key measures used in this analysis to assess the journals in which the EPA HHRA papers were published are the Impact Factor and Immediacy Index. The Impact Factor is a measure of the frequency with which the "average article" in a journal has been cited in a particular year. The Impact Factor helps evaluate a journal's relative importance, especially when compared to other journals in the same field. The Immediacy Index is a measure of how quickly the "average article" in a journal is cited. This index indicates how often articles published in a journal are cited within the same year and it is useful in comparing how quickly journals are cited. Thomson Scientific's Web of Science provides access to current and retrospective multidisciplinary information from approximately 8,830 of the most prestigious, high impact research journals in the world. Web of Science also provides cited reference searching. Scopus is a large abstract and citation database of research literature and quality Web sources designed to support the literature research process. Scopus offers access to 15,000 titles from 4,000 different publishers, more than 12,850 academic journals (including coverage of 535 Open Access journals, 750 conference proceedings, and 600 trade publications), 27 million abstracts, 245 million references, 200 million scientific Web pages, and 13 million patent records. 1 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications The report includes a summary of the results of the bibliometric analysis, an assessment of the 292 HHRA journal articles analyzed by ESI field (e.g., Clinical Medicine, Environment/Ecology), an analysis of the journals in which the HHRA papers were published, an assessment of the non-journal publications, a table of the highly cited researchers in the HHRA Research Program, and information on the patents/patent applications (if any) that have resulted from the program. SUMMARY OF RESULTS 1. About one-sixth of the 292 HHRA journal publications are highly cited papers. 48 (16.4%) of the 292 HHRA journal publications qualify as highly cited when using the ESI criteria for the top 10% of highly cited publications. This is 1.6 times the number expected. 3 (1.0%) of the HHRA journal papers qualify as highly cited when using the ES/ criteria for the top 1 %, which is about the number expected. No journal publications qualified as very highly cited when using the criteria for the top 0.1% or top 0.01% thresholds for the most highly cited papers. 2. The HHRA journal publications are more highly cited than the average paper. Using the ES/ average citation rates for papers published by field as the benchmark, in 9 of the 14 fields in which the 292 HHRA journal papers were published, the ratio of actual to expected cites is greater than 1, indicating that the HHRA journal publications are more highly cited than the average papers in those fields. For all 14 fields combined, the ratio of total number of cites to the total number of expected cites (1,897 to 1,592) is 1.2, indicating that the HHRA journal papers are more highly cited than the average paper. 3. Nearly one-half of the HHRA journal papers are published in high impact journals. 120 of the 292 journal papers were published in the top 10% of journals ranked by JCR Impact Factor, representing 41.1% of the HHRA journal publications. This number is 4.1 times higher than expected. 131 of the 292 papers appear in the top 10% of journals ranked by JCR Immediacy Index, representing 44.9% of EPA's HHRA journal publications. This number is 4.5 times higher than expected. 4. There were no hot papers. Using the hot paper thresholds established by ESI as a benchmark, no hot papers were identified in the analysis. Hot papers are papers that are highly cited shortly after they are published. 5. The authors of the HHRA journal publications cite themselves much less than the average author. 111 of the 1,897 cites are author self-cites. This 5.8% author self-citation rate is well below the accepted range of 10-30% author self-citation rate. 6. About 5% of the non-journal publications are highly cited and the non-journal publications are cited less than the average journal publication. 6 (4.7%) of the 128 non-journal publications qualify as highly cited when using the ESI criteria for the top 10% of highly cited publications. This is about one-half the number expected. 2 (1.6%) of the HHRA papers qualify as highly cited when using the ESI criteria for the top 1 %, which is 1.6 times the number expected. None of the non-journal publications qualified as very highly cited when using the criteria for the top 0.1% or top 0.01% thresholds for the most highly cited publications. The ratio of total number of cites to the total number of expected cites (255 to 664) is 0.4, indicating that the HHRA non-journal publications are cited less than the average journal article. 7. Sixteen of the 402 authors of the HHRA journal publications are included in ISIHighlyCited.com, which is a database of the world's most influential researchers who have made key contributions to science and technology during the period from 1981 to 1999. 8. No patents were issued and no patent applications were filed by investigators from 2001 to 2007 for research that was conducted under EPA's HHRA Research Program. ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications Highly Cited HHRA Journal Publications All of the journals covered by ESI are assigned a field, and to compensate for varying citation rates across scientific fields, different thresholds are applied to each field. Thresholds are set to select highly cited papers to be listed in ESI. Different thresholds are set for both field and year of publication. Setting different thresholds for each year allows comparable representation for older and younger papers for each field. The 292 HHRA journal publications reviewed for this analysis were published in journals that were assigned to 17 of the 22 ESI fields. The distribution of the papers among these 17 fields and the number of citations by field are presented in Table 1. Table 1. HHRA Journal Publications by £37 Fields ESI Field Clinical Medicine Environment/Ecology Neuroscience & Behavior Pharmacology & Toxicology Biology & Biochemistry Social Science, General Immunology Agricultural Sciences Geosciences Molecular Biology & Genetics Engineering Chemistry Mathematics Microbiology Computer Science Multidisciplinary Materials Science Total = 17 No. of Citations 592 472 281 237 62 59 56 40 40 29 13 9 6 1 0 0 0 Total = 1,897 No. of EPA HHRA Papers 79 92 17 54 8 14 3 3 5 4 4 3 2 1 1 1 1 Total = 292 Average Cites/Paper 7.5 5.1 16.5 4.4 7.8 4.2 18.7 13.3 8.0 7.2 3.2 3.0 3.0 1.0 0.0 0.0 0.0 6.5 There are 48 (16.4% of the 292 journal papers analyzed) highly cited HHRA journal publications in 9 of the 17 fields—Clinical Medicine, Environment/Ecology, Neuroscience & Behavior, Pharmacology ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications & Toxicology, Immunology, Social Sciences, Agricultural Sciences, Geosciences, and Engineering— when using the ESI criteria for the top 10% of papers. Table 2 shows the number of HHRA journal publications in those 9 fields that meet the top 10% threshold in ESI. Table 2. Number of Highly Cited HHRA Journal Publications by Field (top 10%) ESI Field Clinical Medicine Environment/Ecology Neuroscience & Behavior Pharmacology & Toxicology Immunology Social Sciences, General Agricultural Sciences Geosciences Engineering Total =9 No. of Citations 217 175 141 83 47 47 40 22 11 Total = 783 No. of Papers 10 18 3 8 1 4 2 1 1 Total = 48 Average Cites/Paper 21.7 9.7 47.0 10.4 47.0 11.8 20.0 22.0 11.0 16.3 % of HHRA Papers in Field 12.7% 19.6% 17.6% 14.8% 33.3% 28.6% 66.7% 20.0% 25.0% 16.4% Three (1.0%) of the journal publications analyzed qualify as highly cited when using the ESI criteria for the top 1% of papers. These publications cover two fields—Environment/Ecology and Clinical Medicine. Table 3 shows the 3 papers by field that meet the top 1% threshold in ESI. The citations for these 3 papers are provided in Tables 4 and 5. None of the HHRA journal publications meet the top 0.1% or top 0.01% ESI thresholds for highly cited papers. Table 3. Number of Highly Cited HHRA Journal Publications by Field (top 1%) ESI Field Environment/Ecology Clinical Medicine Total = 2 No. of Citations 26 33 Total = 59 No. of Papers 2 1 Total = 3 Average Cites/Paper 13.0 33.0 19.7 % of HHRA Papers in Field 2.2% 1.3% 1.0% ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications Table 4. Highly Cited HHRA Journal Publications in the Field of Environment/Ecology (top 1%) No. of Cites 20 6 First Author Meng QY Selgrade MK Paper Influence of ambient (outdoor) sources on residential indoor and personal PM2 5 concentrations: analyses of RIOPA data. Journal of Exposure Analysis and Environmental Epidemiology 2005; 15(1): 17- 28. Induction of asthma and the environment: what we know and need to know. Environmental Health Perspectives 2006;114(4):615-619 Fable 5. Highly Cited HHRA Journal Publications in the Field of Clinical Medicine (top 1% No. of Cites 33 First Author KelloffG Paper Progress and promise of FDG-PET imaging for cancer patient management and oncologic drug development. Clinical Cancer Research 2005;11(8):2785-2808. Ratio of Actual Cites to Expected Citation Rates The expected citation rate is the average number of cites that a paper published in the same journal in the same year and of the same document type (article, review, editorial, etc.) has received from the year of publication to the present. Using the ESI average citation rates for papers published by field as the benchmark, in 9 of the 17 fields in which the EPA HHRA journal papers were published, the ratio of actual to expected cites is greater than 1, indicating that the HHRA journal publications are more highly cited than the average papers in those fields (see Table 6). For all 17 fields combined, the ratio of total number of cites to the total number of expected cites (1,897 to 1,594) is 1.2, indicating that the HHRA journal publications are more highly cited than the average paper. Table 6. Ratio of Actual Cites to Expected Cites for HHRA Journal Publications by Field ESI Field Agricultural Sciences Biology & Biochemistry Chemistry Clinical Medicine Computer Science Total Cites 40 62 9 592 0 Expected Cite Rate 8.1 99.7 16.2 446.9 0.2 Ratio 4.9 0.6 0.6 1.3 0.0 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications ESI Field Engineering Environment/Ecology Geosciences Immunology Materials Science Mathematics Microbiology Molecular Biology & Genetics Multidisciplinary Neuroscience & Behavior Pharmacology & Toxicology Social Sciences, General TOTAL Total Cites 13 472 40 56 0 6 1 29 0 281 237 59 1,897 Expected Cite Rate 5.4 406.5 15.8 43.6 0.0 3.7 11.4 42.1 1.8 191.9 274.8 26.0 1,594.1 Ratio 2.4 1.2 2.5 1.3 *1.0 1.6 0.1 0.7 0.0 1.5 0.9 2.3 1.2 * The actual number of cites is equal to the expected number of cites, making the ratio 1.0. JCR Benchmarks Impact Factor. The JCR Impact Factor is a well known metric in citation analysis. It is a measure of the frequency with which the "average article" in a journal has been cited in a particular year. The Impact Factor helps evaluate a journal's relative importance, especially when compared to others in the same field. The Impact Factor is calculated by dividing the number of citations in the current year to articles published in the 2 previous years by the total number of articles published in the 2 previous years. Table 7 indicates the number of HHRA journal publications published in the top 10% of journals, based on the JCR Impact Factor. One hundred twenty (120) of 292 journal papers were published in the top 10% of journals, representing 41.1% of EPA's FIFIRA journal publications. This indicates that nearly one-half of the FIFIRA journal publications are published in the highest quality journals as determined by the JCR Impact Factor, which is 4.1 times higher than the expected percentage. Table 7. HHRA Journal Publications in Top 10% of Journals by JCR Impact Factor EPA HHRA Papers in that Journal 1 Journal New England Journal of Medicine Impact Factor (IF) 51.296 JCR IF Rank 2 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications EPA HHRA Papers in that Journal 2 1 2 1 2 2 1 1 4 31 1 2 5 1 1 2 8 1 1 1 7 1 2 1 1 7 1 2 Journal Journal of the National Cancer Institute Journal of Allergy and Clinical Immunology Annals of Neurology Development Arthritis and Rheumatism Cancer Research Mutation Research - Reviews in Mutation Research FASEB Journal Clinical Cancer Research Environmental Health Perspectives Journal of Biological Chemistry Neurology American Journal of Epidemiology Molecular Cancer Therapeutics Emerging Infectious Diseases Pediatrics Toxicology and Applied Pharmacology Breast Cancer Research and Treatment Cancer International Journal of Epidemiology Epidemiology Global Change Biology Cancer Epidemiology Biomarkers & Prevention Mutation Research - Fundamental and Molecular Mechanisms of Mutagenesis Rheumatology Environmental Science & Technology Autoimmunity Reviews Critical Reviews in Toxicology Impact Factor (IF) 15.27 8.829 8.051 7.764 7.751 7.656 7.579 6.721 6.177 5.861 5.808 5.690 5.241 5.137 5.094 5.012 4.722 4.671 4.582 4.517 4.339 4.339 4.289 4.111 4.052 4.040 3.760 3.707 JCR IF Rank 44 136 154 165 166 172 175 206 228 255 260 270 308 326 332 345 397 401 413 424 452 452 463 505 516 518 603 623 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications EPA HHRA Papers in that Journal 2 1 17 2 1 3 1 Total = 120 Journal Journal of Neurophysiology BMC Bioinformatics Toxicological Sciences Biology of Reproduction Neuroscience Cancer Letters Journal of Endocrinology Impact Factor (IF) 3.652 3.617 3.598 3.498 3.427 3.277 3.072 JCR IF Rank 645 656 662 694 721 111 853 Immediacy Index. The JCR Immediacy Index is a measure of how quickly the average article in a journal is cited. It indicates how often articles published in a journal are cited within the year they are published. The Immediacy Index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year. Table 8 indicates the number of HHRA journal publications published in the top 10% of journals, based on the JCR Immediacy Index. One hundred thirty-one (131) of the 292 papers appear in the top 10% of journals, representing 44.9% of the HHRA journal papers. This indicates that nearly half of the HHRA journal papers are published in the highest quality journals as determined by the JCR Immediacy Index, which is 4.5 times higher than the expected percentage. Table 8. HHRA Journal Publications in Top 10% of Journals by JCR Immediacy Index EPA HHRA Papers in that Journal 1 2 2 1 1 1 7 1 Journal New England Journal of Medicine Journal of the National Cancer Institute Annals of Neurology International Journal of Epidemiology Journal of Allergy and Clinical Immunology Development Epidemiology FASEB Journal Immediacy Index (II) 12.743 2.776 2.716 2.200 1.790 1.579 1.437 1.241 JCR 11 Rank 2 58 61 84 118 157 187 238 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications EPA HHRA Papers in that Journal 2 2 2 1 2 5 1 4 30 2 2 2 2 17 1 1 1 1 3 7 3 1 1 1 1 13 1 1 Journal Emerging Infectious Diseases Cancer Research Arthritis and Rheumatism Journal of Biological Chemistry Neurology American Journal of Epidemiology Mutation Research - Reviews in Mutation Research Clinical Cancer Research Environmental Health Perspectives Critical Reviews in Toxicology Journal of Neurophysiology Pediatrics Biology of Reproduction Toxicological Sciences Cancer Rheumatology Omics - A Journal of Integrative Biology Global Change Biology Cancer Letters Environmental Science & Technology Journal of Rheumatology Cancer Chemotherapy and Pharmacology Bioorganic & Medicinal Chemistry Letters Neuroscience Thrombosis Research Journal of Exposure Science and Environmental Epidemiology Environmental Research Molecular Carcinogenesis Immediacy Index (II) 1.222 1.220 1.204 1.110 1.110 1.091 1.050 1.010 0.994 0.880 0.821 0.784 0.736 0.734 0.713 0.698 0.694 0.660 0.658 0.646 0.637 0.631 0.624 0.611 0.605 0.596 0.583 0.580 JCRll Rank 243 246 251 291 291 306 331 354 373 442 500 537 593 597 629 649 657 705 707 729 746 755 769 790 801 821 844 852 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications EPA HHRA Papers in that Journal 1 2 1 1 Total = 131 Journal Molecular Cancer Therapeutics Cancer Epidemiology Biomarkers & Prevention Clinical Journal of the American Society of Nephrology BMC Bioinformatics Immediacy Index (II) 0.568 0.560 0.558 0.557 JCRll Rank 877 889 897 899 Hot Papers ESI establishes citation thresholds for hot papers, which are selected from the highly cited papers in different fields, but the time frame for citing and cited papers is much shorter—papers must be cited within 2 years of publication and the citations must occur in a 2-month time period. Papers are assigned to 2-month periods and thresholds are set for each period and field to select 0.1% of papers. Using the hot paper thresholds established by ESI as a benchmark, there were no hot papers identified among the 292 journal publications included in this analysis. Author Self-Citation Self-citations are journal article references to articles from that same author (i.e., the first author). Because higher author self-citation rates can inflate the number of citations, the author self-citation rate was calculated for the HHRA papers. Of the 1,897 total cites of the 292 journal publications, 111 are author self-cites—a 5.8% author self-citation rate. Garfield and Sher3 found that authors working in research-based disciplines tend to cite themselves on the average of 20% of the time. MacRoberts and MacRoberts4 claim that approximately 10-30% of all the citations listed fall into the category of author self-citation. Kovacic and Misak5 reported a 20% author self-citation rate for medical literature. Therefore, the 5.8% self-cite rate for the HHRA papers is well below the range for author self-citation. Non-Journal Publications This analysis included 128 non-journal publications (e.g., technical reports, books/book chapters, handbooks). For the non-journal publications, the Cited Reference Search feature of Web of Science was used to determine the number of times the non-journal publications were cited in the journals Garfield E, Sher IH. New factors in the evaluation of scientific literature through citation indexing. American Documentation 1963;18(July):195-210. 4 MacRoberts MH, MacRoberts BR. Problems of citation analysis: a critical review. Journal of the American Society of Information Science 1989;40(5):342-349. 5 Kovacic N, Misak A. Author self-citation in medical literature. Canadian Medical Association Journal 2004;170(13):1929-1930. 10 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications covered by Web of Science. Through an iterative search process, citing references were identified, duplicates were eliminated, and times cited counts were generated. The times cited data for the non- journal publications in this report do not include the citations of these documents in other non-journal publications. Of the 128 non-journal publications, 39 (30.5%) had been cited at least once in a journal. Each non- journal publication was assigned an ESI field so that the ESI thresholds could be used as a baseline for identifying highly cited publications. The non-journal publications covered 9 of the 22 ESI fields. There are 6 (4.7% of the papers analyzed) highly cited HHRA non-journal publications in 1 of the 9 ESI fields—Environment/Ecology—when using the ESI criteria for the top 10% of papers. Two (1.6%) of the non-journal publications analyzed qualify as highly cited when using the ESI criteria for the top 1% of publications. These publications are in the field of Environment/Ecology. Table 9 shows the 6 non-journal publications that meet the top 10% threshold in ESI as well as the two publications that meet the top 1% threshold in ESI. None of the non-journal publications meet the ESI criteria for the top 0.1% or top 0.01%. The 128 non-journal publications were cited 255 times. The ratio of the total number of cites to the total number of expected cites (calculated using the ESI average citation rates for publications by field as the benchmark) is 0.4 (255 to 664), indicating that the HHRA journal publications are cited less than the average journal article. Table 9. Highly Cited HHRA Non-Journal Publications (top 10%) ESI Field Environment/Ecology Environment/Ecology Environment/Ecology Environment/Ecology Environment/Ecology Environment/Ecology No. of Cites 25 *70 18 24 *25 10 Publication Child-Specific Exposure Factors Handbook (Interim Report). U.S. Environmental Protection Agency, EPA 600/P-00/002B, 2002. Health Assessment Document for Diesel Engine Exhaust. U.S. Environmental Protection Agency, EPA 600/8-90/057F, 2002. An Examination of EPA Risk Assessment Principles and Practices. U.S. Environmental Protection Agency, EPA 100/B- 04/001,2004. Air Quality Criteria for Particulate Matter. Volume I and Volume II. U.S. Environmental Protection Agency, EPA 600/P- 99/002aF,bF, 2004. Guidelines for Carcinogen Risk Assessment. U.S. Environmental Protection Agency, EPA 930/P-03/001F, 2005. Supplemental Guidance for Assessing Susceptibility from Early- Life Exposure to Carcinogens. U.S. Environmental Protection Agency, EPA 630/R-03/003F, 2005. * These are the two non-journal publications that qualify as highly cited when using the ESI criteria for the top 1% of publications. 11 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications Highly Cited Researchers A search of Thomson's ISIHighlyCited. com revealed that 16 (4.0%) of the 402 authors of the HHRA papers are highly cited researchers. ISIHighlyCited.com is a database of the world's most influential researchers who have made key contributions to science and technology during the period from 1981 to 1999. The highly cited researchers identified during this analysis of the HHRA publications are presented in Table 10. Table 10. Highl Highly Cited Researcher Andersen, Melvin E. Birnbaum, Linda S. Brown, John S. Giesy, John P. Goldman, Lee Guillette, Louis J. Klaassen, Curtis Dean Lioy, Paul J. Liu, Jie Lovley, Derek R. Needham, Larry L. Peterson, Richard E. Richards, James H. Schwartz, Joel David Watson, John G. Winer, Arthur M. Total = 16 y Cited Researchers Authoring HHRA Journal Publications Affiliation Chemical Industry Institute of Toxicology (CUT) U.S. EPA, National Health & Environmental Effects Research Laboratory Battelle Coastal Resources and Environmental Management Michigan State University University of California, San Francisco University of Florida University of Kansas Medical Center University of Medicine & Dentistry of New Jersey National Cancer Institute University of Massachusetts Amherst Centers for Disease Control and Prevention, National Center for Environment Health University of Wisconsin, Madison University of California, Davis Harvard School of Public Health Desert Research Institute University of California, Los Angeles ESI Field Pharmacology Pharmacology Environment/Ecology Environment/Ecology Clinical Medicine Environment/Ecology Pharmacology Environment/Ecology Pharmacology Microbiology Environment/Ecology Pharmacology Environment/Ecology Environment/Ecology Environment/Ecology Environment/Ecology 12 ------- Bibliometric Analysis of Human Health Risk Assessment Research Program Publications Patents No patents have been issued or patent applications filed by investigators from 2001 to 2007 for research that was conducted under EPA's HHRA Research Program. This bibliometric analysis was prepared by Beverly Campbell of The Scientific Consulting Group, Inc. in Gaithersburg, Maryland under EPA Contract No. EP-C-05-015 13 ------- |