Research Metrics

Research Metrics is a broad term that encompasses various mathematically calculated measures that demonstrate the impact of scholarly outputs, as well as authors, institutions, disciplines, and journals. Individual metrics often rely on quantitative criteria that are a proxy for impact (for example, the journal impact factor metric uses the number of citations as a proxy for impact). 

If you have questions or concerns about research metrics, book an appointment with us. 

We can help you:

• understand the role of each type of research metric  
• use research metrics to understand when and how others are engaging with your research 
• use research metrics to bolster your grant application 
• use research metrics to articulate your impact as part of career milestones like applying for promotion and tenure, or for awards 

What factors influence the usefulness of research metrics? 

• Discipline and sub-discipline - there is a significant amount of variation between disciplines in the types of materials published, the number of publications, citation practices, collaboration practices. For example, in the health sciences publications are often published more rapidly, with more collaborators, and cited earlier in their life than in the humanities 
• Researcher’s length of career - researchers who are well-established tend to be more highly cited than early-career researchers, regardless of the quality of the publication.
• Reason for citation - articles may be cited for a variety of reasons including, review of interest in a research topic, to support an argument, or to remark on the poor quality of methods or analysis 
• The citation tracking database being used - the number of citations may appear to change, but it’s because the database’s scope of indexed materials has changed. No citation database is comprehensive. Many databases, such as Web of Science or Scopus for example, are proprietary which means that to get an accurate picture of citations you may need to consult multiple sources 
• Bias - research suggests that research metrics are not free from bias, including with regards to gender and geographic location.  
• Data Errors - misspellings of author names, author names that change, or incorrect attribution for authors that share a name can all impact metrics. ORCIDs can help disambiguate authors. 

What types of research metrics are there? 

The most common types of research metrics are citation based, and can be assessed at a journal-level, article-level, or author-level. Other research metrics consider non-citation sources of data and these are referred to as alternative metrics or altmetrics. There are also non-mathematically calculated measurements that can be used when demonstrating research impact, including peer review, awards, funding, patents or intellectual property, and broader research impact frameworks that can help create narratives of social and scholarly impact.  

Selecting and using research metrics 

We recommend approaching all research metrics with caution for a few reasons. One, there is no completely accurate way to measure the impact of work mathematically. Two, many of these metrics are developed and maintained by for-profit companies that do not index everything, leading to legitimate and impactful publishing being left out of their calculations. This is especially true for open access and non-English language publications. 

Because of the limitations of each, we recommended that you apply them thoughtfully, and use multiple metrics to present a more holistic picture of the importance of the research which is being done. 

Journal metrics 

Journal metrics are intended to demonstrate the impact level or performance of a journal by using citation analysis. The usefulness of citations as a proxy for impact is affected by disciplinary norms, reason for citation (could be to cast the research in a negative light), and the citation database being used. Journal-level metrics can also be impacted by having a few “superstar” articles that increase the impact for the entire journal, the length of time that a journal has been publishing, and the inability to compare between disciplines based on variations in citation practices. 

It is important to remember that past success of articles within a journal does not predict future success, so use caution when relying on journal metrics.  

When to use journal metrics 

• To compare journals that publish in the same discipline 
• To identify which journal to submit to.  
• To learn more about the corpus of publications in your field, such as journal scope and the availability of open access publications 

Examples of journal metrics 

• Journal Impact Factor (JIF) - looks at the ratio of citations to articles in the journal relative to the number of citable publications in that journal over a two-year or five-year period. JIF is calculated by Clarivate using data from their proprietary Web of Science database, and what is determined to be a citable publication is negotiated between publishers and Clarivate. 
• Eigenfactor Score – looks at the number of citations over a five-year period using the same Clarivate Web of Science data as JIF. However, it weights citations from journals that are themselves highly cited more heavily than citations from journals that are less cited, and removes “journal self-citation” (citations of articles in the journal to other articles in that journal) 
• CiteScore (CS) - looks at the number of citations to articles in the journal relative to the number of publications in that journal over a four-year period. CS is calculated by Elsevier using data from their proprietary Scopus database (U of G does not currently subscribe to Scopus). 
• SCImago Journal Rank (SJR) - looks at the number of citations over a three-year period using the same Elsevier Scopus data as CS. Similar to the Eigenfactor score, it weights citations from journals that are themselves highly cited more heavily than citations from journals that are less cited. It allows some “journal self-citation” but does not allow for it to account for more than one third of all incoming citations. 
• Immediacy Index – the length of time it takes for articles in a journal to be cited. Available through Journal Citation Reports from Clarivate. 

Where can I find journal metrics? 

There are two main locations where you can find journal metrics: directly on journal publisher websites, and on the websites of companies/organizations which calculate the metrics.  

• Journal Citation Reports (JCR) is a resource paid for by the library that contains Journal Impact Factor (JIF). This tool can be used to compare JIF, total citations, and percentage of open access content. The Eigenfactor score can be seen by selecting a journal and scrolling down to “Additional Metrics” at the bottom of the page. 
• Scopus CiteScore is publicly available and contains the CiteScore (CS). This website can be used to compare CS and total citations across journals. The ability to filter to Open Access journals is also present. 
• SCImago Journal Rank is publicly available and contains the SCImago Journal Rank (SJR). It can compare the SJR, citations and references across journals. The ability to filter to Open Access journals is also present.

Article metrics 

Article level metrics attempt to quantify the impact of individual articles, usually by using citations. Other article level metrics consider other sources of data and these are referred to as alternative metrics or altmetrics.  

Examples of article metrics 

• Citation count – the number of times an article has been cited 
• Field Normalized Citation Impact (FNCI) - calculated as the ratio between the citations received by an article and the average number of citations received by all other similar publications within a particular database. Two common field normalized citation metrics types include the Relative Citation Ratio (RCR) from inCites (Clarivate) and Field Weighted Citation Impact (FWCI) from Scopus (Elsevier). 
• Citation Percentiles and Rankings – demonstrates the position of an article compared to other articles in the same discipline, country and/or time period based on the number of citations they have received. SciVal and Scopus both calculate their rankings based on the Scopus database, and inCite uses Clarivate’s Web of Science database. 

Where can I find article metrics?

Article citation counts are often visible on journal websites, in abstract and index databases, in repositories, and through proprietary journal article databases, which you can find on the Library’s A-Z Database List. 

Author metrics 

Author metrics attempt to determine the impact of an author’s research, using publication counts and citation counts. Author metrics should be used for individual understanding of your own publications and should not be used to compare researchers because of the known impacts of bias in how researchers are cited based on gender and other identities. 

Examples of author metrics 

• h-index – the highest number of publications with at least that number of citations. For example: a researcher has an h-index of seven if they have seven publications with at least seven citations. 
• i10 Index – the number of publications written by the author that have been cited at least 10 times. It is only used by Google Scholar. 

Alternative metrics (altmetrics) 

Alternative metrics, often referred to as altmetrics, take into consideration measures of impact other than citations in scholarly journals.  

Alternative metrics can provide context for how work is being used by the scholarly community and general public, but they are also susceptible to manipulation and cannot tell the whole story of how impactful your research is. They are used inconsistently across the scholarly landscape. 

Examples of alternative metrics 

• Altmetric – monitors online attention to research with a variety of data sources, visualized as a colourful “altmetric donut”. Integrated into various databases and journals. 
• ImpactStory / Our Research – a tool used to aggregate alternative metrics of engagement with research. Requires users to create a profile and draws primarily from Altmetric for their data sources. 
• PlumX – a suite of alternative metrics products that focus on usage, mentions, social media, citations and captures. PlumX is available as a subscription service and is also integrated into various databases and journal websites. It draws on a wide variety of data sources. 

Get assistance 

Individual appointments  

Book an individual appointment for support with any of the above topics.

In-class instruction 

We can support your students by providing in-class workshops (virtually or in-person) on any of the topics listed above. Connect with us to discuss library research support for your students.