A study shows that negative citations of papers are rather rare. How do they work exactly?

Isn’t science the modern great art of debate? The only way through which science can be academically validated is by passing through a long process, often problematic for the author(s), in which the study must be thoroughly analysed and dissected by other experts in the same discipline in order to point out its weaknesses; a process called “peer-review”.

Furthermore, after the publication the work (and its authors) will have to face other authors and studies that will cite it in the attempt to challenge its findings and data. Or do they?

A study led by Alexander Oettl at the Georgia Institute of Technology in Atlanta and published on 26 October 2015 in the journal Proceedings of the National Academy of Sciences shows that negative citations, considered as a professional and formal disagreement, are quite rare. Furthermore, negative citations usually address only the more famous and highly-cited studies.

Counting how many times a paper is cited by other studies is probably the most straightforward and catchy way to have an idea of its importance and, in other words, its impact on the field. But are citations just a way for acknowledging the excellence of a work or a tool for stimulating a professional debate?

Oettl and his team analysed some 762,000 citations from 16,000 articles published in the Journal of Immunology referring to almost 150,000 papers, manually sorting between positive and negative citations. They found that only 2.4% of the citations could be considered negative. “They can be attempting to constrain results, note inconsistencies with other research, point out statistical flaws or correct other issues”, said Oettl.

Interestingly, negative citations usually come from researchers working in the same discipline (which is good and is supposed to keep the quality level of the debate high), but “geographically distant” from the author they are criticising. As explained by another author of the study, Nicola Lacetera of the University of Toronto’s Institute for Management and Innovation, the latter aspect shows how social factors play a role in the professional debate within science. You wouldn’t criticise one of your colleagues with whom you might collaborate in the future, would you?

Lastly, Michael Schreiber from the Technical University of Chemnitz, Germany, thinks that the definition of negative citation used in this study might be too broad since it includes negative self-citations largely used by authors to underline the weaknesses of their own previous studies compared to the “striking” new ones.

peer review
Peer review, from flickr.

Thus citing negatively (but appropriately) is more complicated than it might appear. “There could be reputational harm from making these negative citations”, observes Oettl, “and if your criticism turns out to be false, this could heavily impact your reputation”.

Critical debates are fundamental in order to rely on scientific community and its studies. However, neither science seems to be immune from social factors and their dynamics.