Understanding publication metrics: impact factor and h-index

I was looking for some papers in Google Scholar yesterday and accidentally clicked on ‘Metrics’ icon located on the right side of the page. In front of me was a list of top 100 journals (of the world) based on their h-index. I was surprised to see that most of the journals in the list were in the field of biological sciences, physics, and chemistry. To my dismay, I didn’t see the journal ‘Nature Geosciences’, the most revered (sorry if you don’t like to say so) journal in the field of Geosciences. But do the above results imply that only the ladies and gentlemen in biology, physics, and chemistry publish good research? Does that mean Geo-scientists watch movies all the time?

The answer is certainly “no”.

So the question is:

Why the journals in biology, physics, and chemistry have the highest impact factor? 

To answer this question, we have to answer another question first:

What exactly is an h-index?

According to Wikipedia, “A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np − h) papers have no more than h citations each.”

Little confused? Very reasonable! But it is very simple.

If a scientist has an h-index of 12, it means that he/she has published 12 papers that received 12 or more citations.

The h-index was proposed by physicist Jorge E. Hirsh in 2005 aiming to measure both the productivity and citation impact.

So to maintain a high h-index, you have to keep publishing while maintaining the quality of your publications (so that you receive more citations). If you publish a paper that is eventually not cited, you are wasting your time as it won’t contribute to your h-index no matter how many more such papers you publish. The h-index for a journal also means the same thing.

Now we are in a situation to answer the original question. If a journal has a high h-index, it generally means that it has published higher number of papers that received higher number of citations. But what does that really mean? It simply means that there are more people working in that field so that the probability of number of papers being published (and cited) is higher.

Still doesn’t make sense? Lets look at this statistics here about the number of PhDs awarded in 2008 (of course in the US). The number of doctorates awarded in biological sciences is 7,793 while it is 862 for earth, atmospheric, and ocean sciences. So obviously we can expect more publications in biological sciences, and hence more citations, and hence higher h-index, in the journals of that field. This is not surprising considering the fact that the history of research in biological sciences, physics, and chemistry is much longer than that of Geosciences. I hope I answered the original question.

Unfortunately, h-index is not perfect as is this world. For example, if a scientist published only one paper in his life time that received say 2,000 citations, his h-index is 1. Another scientist who published one paper and received only 15 citations, would still have an h-index of 1. Another drawback of these metrics including the h-index is that they do not reflect the difference among different fields. So the h-index of a journal/professor of biology cannot be compared to the h-index of a professor/journal of Geosciences.

In your Google Scholar profile, you will also see another index called i10-index developed by google, which, in fact, is very similar to the h-index. It is just the number of your papers that received at least 10 citations.

Another frequently asked question:

Do you really need to publish in a journal having a very high impact factor to prove yourself?

The answer is “not really”. Have you realized that the impact factor of your paper may be higher or lower than the impact factor of the journal itself?

According to Wikipedia, the impact factor of a journal in any given year is the average number of citations received per paper published in that journal during the two preceding years. So if a journal has a high impact factor, either the journal received higher number of citations or the journal accepted fewer number of articles, or it could be both. For most of the journals, the acceptance ratio would not be too much different. So the major component affecting the impact factor is the number of citations indeed.

Lets take an example of a paper published in 2013. This paper published in April 2013 has received 25 citations until July 2015. That means the impact factor of this paper is 25. Note that the impact factor of the journal Nature in which this paper was published is 41.45 in 2014. So it can be said that this paper is significantly below average for this journal.

Lets take another contrasting example. This paper has received 50 citations in a similar time period, i.e., between May 2013 to July 2015. This means that the impact factor of this paper is 50. Now compare this to the impact factor of the JGR-Atmosphere journal in which it was published, i.e., 3.43. It means that this paper has significantly higher impact than the journal’s average impact.

So it really doesn’t matter in which journal you publish your paper. The only thing that matters is whether you receive citations or not.

One last question:

What is the best way to measure productivity of a researcher?

There will probably be many indices rolling in the future each claiming that it is the best. But, perhaps the best way to evaluate a researcher’s productivity or performance is to simply look at both the number of citations and the number of publications, both of which are shown by Google Scholar and other academic search engines (if made public). In addition, quality of the paper must be evaluated by a researcher in that field. It is that simple. I am not saying that we don’t need those indices. When making comparison among many candidates, those indices are indeed very helpful. And it is always fun to compare your index with that of your peers in Google Scholar. 🙂

Thank you for reading.

P.S.: The original paper about the h-index can be found here.

The author is thankful to Travis Swanson for helpful discussion.