​

Hello fellow journalologists,

This week I’d like to try something different. In previous newsletters I’ve tended to riff off a recent news event and provide a perspective on what the development might mean for journal editors and publishers.

In this newsletter I want to show you how to use Digital Science’s Dimensions to create article segmentation within journals. This can be a powerful method for assessing a journal’s performance against its competitors.

I only started using Dimensions relatively recently and I’m becoming more impressed with it over time. The user interface takes some getting used to, but it’s a powerful tool that I’m using regularly in my consultancy work.

One of the filters within Dimensions relates to “Research Categories”, which are used to classify the subject area of individual articles within the database. The categorisation schemes are:

• Fields of Research (ANZSRC 2020)
• Research, Condition, and Disease Categorization (RCDC)
• Health Research Classification System (HRCS)
• Broad Research Areas (BRA)
• Health Research Areas (HRA)
• Common Scientific Outline (CSO) and Cancer Type codes
• Units of Assessment (UoA)
• Sustainable Development Goals (SDGs)

Today I’m going to provide some case studies using the two schemes that are in bold font.

Broad Research Areas

Let’s start off with Broad Research Areas, which Dimensions defines as follows:

This classification scheme is helpful because it is so simple; it allows us to segment a medical journal’s content to determine whether it's serving bench scientists, clinical researchers, or public-health experts. Furthermore, by measuring the proportion of papers that are allocated to each classification, we can see how a journal’s editorial scope changes over time.

Nature Medicine was launched in 1995 and for the first two decades had a scope that fell under the umbrella of “translational medicine”. Most of the articles it published described experiments using animal models; despite its name, Nature Medicine published very little clinical content. In December 2017 the Nature journals editorial management team hired Joao Monteiro to be Chief Editor. Joao is an MD PhD and had a bold vision to make Nature Medicine publish, well… medicine.

That’s easier said than done, even for a journal backed by the Nature imprimatur. After all, journals such the New England Journal of Medicine (first published in 1812) and The Lancet (1823) were launched before Nature (1869) and are incredibly strong brands within medicine.

Let’s see what happened when Joao took over as Chief Editor, using the Broad Research Area classification as a guide.

The proportion of ‘basic science’ dropped from about 75% in 2013 to 20% in 2022. In its place, the journal started to publish work that Dimensions classifies as “clinical medicine and science” and “public health”.

Nature Medicine’s direct competitor is Science Translational Medicine, which continues to primarily focus on ‘basic science’. Indeed, it looks like there has been a slight increase in the proportion of ‘basic science’ that the journal has published over the past decade, as you can see in the graph below.

I’ve created similar graphs for the other leading general medical journals: NEJM, The Lancet, JAMA, BMJ, and PLOS Medicine. The results are unsurprising if you work in this area, but are interesting nonetheless. I’ll probably post those graphs on LinkedIn next week.

Units of Assessment (UoA)

OK, let’s now take a look at another classification scheme, which Dimensions defines as follows:

This scheme can help to classify the content in broad scope journals, allowing us to see which subject areas are strengths and weaknesses for a journal compared with its competitors.

The tables that follow list the number of articles, published in 2020 and 2021, that were assigned to each Unit of Assessment. For brevity I’ve omitted categories outside of STEM.

As far as I can tell, Dimensions assigns categories to every paper that has sufficient content to let the algorithm do its magic, so review and opinion articles are probably included. Let’s kick off by taking a look at Nature and Science.

The number of articles in each subject area are comparable. Science seems to publish more content in some areas (e.g. chemistry and engineering), but the numbers are remarkably similar.

The picture is very different when we compare Nature Communications with Science Advances:

Nature Communications publishes many more articles each year than Science Advances, but even when you take that into account there are some areas where Nature Communications has a clear advantage (e.g. chemistry, which is slightly surprising since it’s a strength for Science).

(I haven’t presented normalised data because some papers are tagged with more than one category. In other words, I don’t know what the denominator is, so I can’t accurately calculate “the percentage of papers in Nature Communications that are ‘clinical medicine’” for example.)

Finally, let’s take a look at Scientific Reports compared with PLOS ONE:

PLOS ONE has a clear lead in public health, but doesn’t do so well in chemistry, physics or engineering, for example.

It’s also interesting to compare the colours in the heat map within a portfolio. For example, Scientific Reports is not particularly strong in “Biological Sciences”, even though that subject is a strength for Nature Communications, which sits upstream from it in the transfer cascade. This is probably because Communications Biology sits between the two journals in the transfer cascade.

I’m surprised at how many ‘clinical medicine’ papers there are for some of these journals — this might reflect the time period (2020 and 2021) that this analysis covers; perhaps there are a large number of Covid related papers in here.

If you care about how frequently your journal is cited (unlikely, I know), Dimensions provides a helpful set of metrics. The table below shows the median citations for Nature Communications and Science Advances content published in 2020 and 2021 (Dimensions allows users to choose between mean and median citations. Since citation distributions are usually highly skewed, I chose to use the median. After all, who in their right mind would calculate the mean of a ratio of citable items and citations?!?).

This is very helpful because it allows us to compare both number of published articles and average citations in tandem. For example, we can see in the tables above that Science Advances publishes half as many papers in ‘engineering’ as Nature Communications, and is also dropping close to 6 citations per paper. If I was at AAAS I’d be trying to address that (but only if I cared about citations and impact factor...).

Conclusions

I hope this newsletter has given you some inspiration for ways to assess your portfolio against its competitors. Segmenting a journal’s content and comparing the results with competitors’ performance can help to inform publishing and editorial strategy. If a journal is weak in a particular area you might want to signal interest in the topic by:

• commissioning reviews
• sending out more papers for peer review
• adding editorial board members with appropriate expertise
• making a ‘hit list’ of researchers working in the area and wooing them at conferences

If doing bibliometric assessments feels a little daunting, then please do get in touch as the Clarke & Esposito team may well be able to help. We spend much of our time helping academic societies to improve their journal portfolio, both in terms of publishing strategy (e.g. transitioning to open access; developing a portfolio strategy by launching journals to create a transfer cascade etc.), but also in terms of measuring and improving editorial performance.

I've decided not to include a “Briefly Quoted” section this week, as this newsletter is already quite long enough! Thank you for reading to the end. If you’d like more of this kind of 'analytics' content, please let me know.

Until next week,

James

P.S. Feel free to share this newsletter with your colleagues if you think they would find it helpful. They can subscribe to the newsletter here:

​https://journalology.ck.page​

​