Citation Analysis and Impact Factors
- Indexing of cited references has permitted not only their use as access points to locate references, but also allowed for citation analysis. See the Eugene Garfield essays linked in Part I of this lecture for more a more thorough discussion of various aspects of citation analysis.
- Of particular importance to researchers is the notion that the more highly cited a given article, or author, or journal or institution is, the greater its impact on scholarly research, and so the more "important" it is. The data in the Web of Science databases has been analyzed in a variety of ways, and used to produce several products based on citation analysis:
- Essential Science Indicators analyzes highly cited papers to identify hot areas of research in 22 fields of research. This tool is especially oriented toward administrators in industry and academia who want to identify potential recruits, new research areas to explore or chart what their competitors are researching. UCSB does not currently subscribe to this database.
- Journal Citation Reports (see below) uses citation data to rank scholarly journals in the sciences and social sciences.
- The most commonly used measure of journal ranking by citation is the impact factor. Generally speaking, it is the ratio of the number of citations of a given journal to the number of articles published in that journal. The higher the number, the greater the "impact" of the journal on its field. More specifically, impact factors are usually measured for a given year. Thus, for the year 2013, the impact factor is calculated as
Impact Factor (2019) = (# of cites in 2014 of articles published in 2017 and 2018) / (# of articles published in 2017 and 2018)
- Impact factors are most often calculated for individual journal titles, but may also be calculated for groups of journals, such as all journals in a given subject category.
- Other factors calculated for Journal Citation Reports include:
- 5-Year Impact Factor - Same as impact factor, but calculated using five years of data, rather than two years of data. Helpful for journals where the pool of articles and cites is small, or to smooth out unusual "bumps" in the data.
- Immediacy Index -- The number of cites in the same year as publication divided by the number of articles published.
- Cited Half-Life -- For a given year, half of the citations to that journal are for articles published after X years.
- Citing Half-Life -- For a given year, half of the articles cited in articles from that journal were published after X years.
- Eigenfactor™ Score - The Eigenfactor was created by Carl T. Bergstrom and colleagues, this measure of journal importance is based on the number of times articles from the journal published in the past five years have been cited in the JCR year, but it also considers which journals have contributed these citations so that highly cited journals will influence the network more than lesser cited journals. References from one article in a journal to another article from the same journal are removed, so that Eigenfactor Scores are not influenced by journal self-citation.
- Article Influence Score - Another Bergstrom creation, it is calculated by dividing a journal's Eigenfactor Score by the number of articles in the journal, normalized as a fraction of all articles in all publications. This measure is roughly analogous to the 5-Year Journal Impact Factor in that it is a ratio of a journal's citation influence to the size of the journal's article contribution over a period of five years. The mean Article Influence Score is 1.00. A score greater than 1.00 indicates that each article in the journal has above-average influence. A score less than 1.00 indicates that each article in the journal has below-average influence.
- Important considerations in using impact factors:
- Impact factors vary dramatically depending on the subject field. Comparing impact factors from journals in two different fields is like comparing apples and oranges. Some fields cite new articles much more heavily than older ones; some cite older articles equaly heavily. Since impact factors are calculated for the most recent two years of citations, journals in the latter fields will tend to have lower impact factors.
- Review articles tend to be more highly cited than original research papers. So, journals which publishe only review articles (e.g., Accounts of Chemical Research, Chemical Reviews) tend to have disproportionately high impact factors compared to other journals in their subject category.
- Since industrial research tends to be reflected in the patent literature more than in the journal literature, articles and journals dealing with applied research tend to be less cited in journals than basic research, and so generate lower impact factors.
- Over time, impact factors tend to rise, due to the growth in overall scholarly research. So, a trend of increasing impact factors for a given journal is only significant when compared to others in its field.
- Remember that journal title changes, splitting and recombining of journal titles, and newly introduced journals may lead to anomalies.
- Remember, too, that sometimes an article may be highly cited because of the number of article published to refute it. Really awful research can sometimes be as highly cited as really good research.
- Uses of impact factors
- Librarians use impact factors to help make decisions on journal purchases and cancellations.
- Researchers use them to help determine where to submit their research for publication.
- Grant-issuing agencies, tenure and promotion committees and agencies issuing work permits ("green cards") use them to evaluate the quality of a researcher's work, assuming that a publication in a high-impact journal is more important than one in a low-impact journal.
- Note that this kind of use of citation analysis is tricky (should self-citation count? do authors and editors try to manipulate citation numbers to their advantage?)
Alternatives to JCR Impact Factors:
- SCImago Journal Rank (SJR) https://www.scimagojr.com/ - SJR is weighted by the prestige of a journal Subject field, quality, and reputation of the journal have a direct effect on the value of a citation. SJR assigns relative scores to all of the sources in a citation network. Its methodology is inspired by the Google PageRank algorithm, in that not all citations are equal. A source transfers its own 'prestige', or status, to another source through the act of citing it. A citation from a source with a relatively high SJR is worth more than a citation from a source with a lower SJR. A source's prestige for a particular year is shared equally over all the citations it makes in that year; this is important because it corrects for the fact that typical citation counts vary widely between subject fields. The SJR of a source in a field with a high likelihood of citing is shared over a lot of citations, so each citation is worth relatively little. The SJR of a source in a field with a low likelihood of citing is shared over few citations, so each citation is worth relatively much. The result is to even out the differences in citation practice between subject fields and facilitate direct comparisons of sources.SJR emphasizes those sources that are used by prestigious titles. SJR allows the user to rank their own customized set of sources, regardless of their subject fields. SJR values are available in the Analytics section of Scopus, based on their data.
- Source Normalized Impact per Paper (SNIP) - SNIP essentially divides the impact factor for a journal by the average impact factor for its field. The idea is to allow one to compare impact across different fields. Scopus Analytics https://www.scopus.com/source/eval.url (requires an account with Elsevier) reports SNIP values based on their data. This calculation can get tricky depending on what subject field(s) a journal may belong to.
- h-index -- This is a tool to rank the work of individual researchers. First proposed by J. E. Hirsch in "An index to quantify an individual's scientific research output", PNAS, 102, 16569-16572 (Nov. 15, 2005), it defines the h-index as the number of papers "h", which each have at least "h" citations in the literature. So, if I have published 600 papers, and the 110 most cited papers have at least 110 citations each, my h-index would be 110. If, on the other hand, I publish 600 papers and only 5 have more than 5 citations, even though those five may have thousands of citations, my h-index would be 5. This index tends to favor researchers who consistent publish large numbers of highly-cited papers, relative to authors who are, perhaps, equally well-regarded but less prolific. Note that Web of Science now can quickly generate h-indices based on their citation data for any set of publications - see Lecture 9.1. Google Scholar Metrics https://scholar.google.com/citations?view_op=top_venues&hl=en has a list of h-indices, grouped by subject area, calculated for journals using citation data from Google Scholar.
- Usage counts -- This measure, increasing in popularity as journal usage shifts more and more into electronic journals, counts the number of times an article is downloaded. This tends to reduce the advantage of basic research over applied research in citation counts, but just because an article was downloaded, was it actually read? Was it considered important by the readers? No way to tell!
- Social Web metrics - Some researchers have begun to measure how often a researcher or an article is referred to in blogs or social networking sites like Facebook and Twitter, or how often the scientist's own blog, tweets, etc. are mentioned in social media. The term "altmetrics" has been coined to describe such social metrics. See "Scholars Seek Better Ways to Track Impact Online" by Jennifer Howard in the Jan. 29, 2012 issue of Chronicle of Higher Education. Some journals are now incorporating altmetric tools, such as PLOS One - display any article and click on the "Metrics" tab for an example. See also "Research impact: Altmetrics make their mark" by Roberta Kwok, originally published in Nature in 2013 See Lecture 17 notes for a more in-depth discussion of altmertics..
- See the article by Sophie L. Rovner, "The Import of Impact", Chemical & Engineering News, 86, 39-42 (May 19, 2008) for a good introductory discussion of the topic.
- Above is the Journal Citation Reports opening screen. At the top left are links to other products in the Web of Science family. At top right are links to sign in (or create a personal account), and Help
- In the middle of the page is a search window, to look for a specific journal title and links to Browse by Journal, Browse by Category and creat Custom Reoorts.
- Clicking the Filter drop down give the options for limiting the journals displayed.
- InCites Journal Citation Reports is Clarivate Analytics' product for journal evaluation based on citation analysis. (For more information, see the training guide at http://clarivate.libguides.com/jcr.
- Above is a screen generated by clicking on the Browse by Journal link. Note the default listing of all indexed journals in alphabetical order.
- Note the Journals by Rank and Categories by Rank tabs at the top of the display. You may display lists of either individual journals, or aggregated data for subject categories.
- Note that there are separate databases for science (including engineering and medicine) and social science journals. See the Select Edition radio buttons in the left-hand column.
- Note also that statistics are generated on a yearly basis, with data generally becoming available in the 3rd quarter of the following year. We currently have access to the 1997-2019 data. The 2020 data should become available around August, 2021. The default is the most recent year. Use the Select JRC Year drop-down menu to select other years.
- You may view groups of journals by subject category, publisher or country of origin, or search for specific journals, or view all journals. See the drop-down menus in the left-hand column. You may select multiple categories, publishers, etc. Also in the left-hand column are options to see journal title changes since the previous year, view open access journals only, or switch from Web of Science journal subject categories to Essential Science Indicators categories.
- You can specify ranges of journals to display: by quartile, by Impact Factor, or by percentage of average impact factor.
- When you select any limiting option on the left=hand side, you must then click the Submit button at the bottom of the column to have it take effect. The Clear button clears all the selected filters, and returns to the default settings.
- Note that any journal may appear in multiple subject categories, and the assignments are not always obvious. Click on the desired category and click the "Submit" button. To select multiple categories (e.g. all Chemistry categories),simply click the check box for each desired category, then "Submit".
- Clicking the "Show Visualization" link displays a graph of the citation connections between the journals or categories displayed. Here's a visualization of the top 25 journals in chemistry by impact factor for 2016. Scrolling up and down on the image zooms in or out.
- Click on Compare Journals to generate bar graphs comparing selected journals over a selected range of years for a selected journal metric. Here are graphs comparing the impact factors of JACS and Nature Chemistry over a five-year period.
- Customize Indicators lets you select which of the metrics will display in the table of journals or categories. See below.
Individual Journal Profile
- Each journal profile gives the full journal title, ISSN, publisher information, standard title abbreviations, subject categories, language of publication, frequency of publication, and links to the journal's table of contents and to the Ulrich's serials information on the journal.
- Note that the default display is for the current years data. However, you can select the two preceding years, or all available years.
- Note the definition of the Journal Impact Factor for the displayed year.
- Below the list of articles, note the tabs. While the default display is Source data, you can also display a box plot, Rank, Cited Journal data, Citing Journal data, and a graphic of Journal Relationships.
- Below that are the Key Indicators, that is, all of the different metrics that JCR calculate; Metric Trend (note the drop-down menu to select which metric is being graphed), and Journal Profile, including a graphic showing how many articles in the journal were published open access.
Browse Categories
Browsing journals by subject category is a useful way to compare impact factors of journals in the same area. Note that these subject categories are the same ones used to categorize journls in Web of Science.
Clicking on a group heading displays a list of all the subject categories in that group.
Clicking on an individual subject category take you through to the list of journals in that category.
© 2022 Charles F. Huber
This work by Charles F. Huber is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Based on a work at guides.library.ucsb.edu
Screenshots of Journal Citation Reports are copyright © 20120 by Clarivate Analytics and are used under fair use for educational purposes only.
