A comment on yesterday's post got me thinking about something: If a non-recent journal article gets a low number of citations (say, 0-2), was the research that went into that paper a waste of time (and money)?
My gut reaction is to say no, of course not. Surely something was learned during the research that led to even the most forgettable or forgotten of papers? And surely the researcher didn't know in advance that the paper would never be cited, and did the research for a good reason?
And perhaps citations are not the most perfect judge of what is or is not worthwhile. It's not difficult to think of examples of highly cited papers that aren't that great, and barely cited papers that are overlooked (especially our own).
At the same time, a paper with zero citations, even after more than 10 years, might mean something..
.. such as:
- no one else in the world is or will be interested in this topic;
- others are interested, but they never publish;
- others are interested, but they only cite other papers, not yours (for various possible reasons), creating a snowball effect of subsequent citation of papers other than yours on this topic. With time, it becomes ever less likely that your paper will be cited.
Publishing something widely believed to be wrong or stupid isn't necessarily a barrier to citations, nor is publishing something obvious, so I am not including these in my list of possibilities.
Since I am in a quantitative mood this week, I tried to decide whether there is a minimum number of citations, above which we can say that the research was worthwhile, and below which we might have good reasons to doubt this.
My musings on this topic made me dive into my citation index to look at some of my low-citation papers to see if I could reasonably defend them as worthwhile in some way. My favorite example of a deservedly ignored paper in my oeuvre has surprised me by being cited in the low double-digits. Does that mean that the paper is more worthwhile than it was a few years ago because it has now received (slightly) more than 10 citations (none by me!) instead of 2 (or zero)? No, I don't think so. The difference between 14 citations and 2 citations really isn't that significant in terms of gauging the worth of a paper. And yet, although I am well aware that it was a fairly insignificant paper, I am reluctant to say it was a waste of time.
Further rummaging in my citation history shows me that some of my most highly cited papers do not represent what I consider to be my most significant work but that happened to be on topics that are of more widespread interest than the core of my research. Does that make these more-cited papers more "important" than my others? I am not objective about this, but I don't believe that citations correlate with significance, though I admit that it depends on how you define "important" and "significance".
One more personal example: A paper that has received a very modest number of citations is frequently mentioned to me as a paper that is read and discussed in graduate seminars. I am very pleased about that. The paper is being read and used (perhaps as an example of how not to write a paper..), although it is not cited very often. I consider that paper to have been worthwhile.
So, although I agree that zero citations is not a good thing for non-recent papers, and my papers have thus far avoided this fate (though in some cases not for any good reason), I have trouble casting aspersions on papers that have received a modest but non-zero number of citations.
My gut reaction is to say no, of course not. Surely something was learned during the research that led to even the most forgettable or forgotten of papers? And surely the researcher didn't know in advance that the paper would never be cited, and did the research for a good reason?
And perhaps citations are not the most perfect judge of what is or is not worthwhile. It's not difficult to think of examples of highly cited papers that aren't that great, and barely cited papers that are overlooked (especially our own).
At the same time, a paper with zero citations, even after more than 10 years, might mean something..
.. such as:
- no one else in the world is or will be interested in this topic;
- others are interested, but they never publish;
- others are interested, but they only cite other papers, not yours (for various possible reasons), creating a snowball effect of subsequent citation of papers other than yours on this topic. With time, it becomes ever less likely that your paper will be cited.
Publishing something widely believed to be wrong or stupid isn't necessarily a barrier to citations, nor is publishing something obvious, so I am not including these in my list of possibilities.
Since I am in a quantitative mood this week, I tried to decide whether there is a minimum number of citations, above which we can say that the research was worthwhile, and below which we might have good reasons to doubt this.
My musings on this topic made me dive into my citation index to look at some of my low-citation papers to see if I could reasonably defend them as worthwhile in some way. My favorite example of a deservedly ignored paper in my oeuvre has surprised me by being cited in the low double-digits. Does that mean that the paper is more worthwhile than it was a few years ago because it has now received (slightly) more than 10 citations (none by me!) instead of 2 (or zero)? No, I don't think so. The difference between 14 citations and 2 citations really isn't that significant in terms of gauging the worth of a paper. And yet, although I am well aware that it was a fairly insignificant paper, I am reluctant to say it was a waste of time.
Further rummaging in my citation history shows me that some of my most highly cited papers do not represent what I consider to be my most significant work but that happened to be on topics that are of more widespread interest than the core of my research. Does that make these more-cited papers more "important" than my others? I am not objective about this, but I don't believe that citations correlate with significance, though I admit that it depends on how you define "important" and "significance".
One more personal example: A paper that has received a very modest number of citations is frequently mentioned to me as a paper that is read and discussed in graduate seminars. I am very pleased about that. The paper is being read and used (perhaps as an example of how not to write a paper..), although it is not cited very often. I consider that paper to have been worthwhile.
So, although I agree that zero citations is not a good thing for non-recent papers, and my papers have thus far avoided this fate (though in some cases not for any good reason), I have trouble casting aspersions on papers that have received a modest but non-zero number of citations.