Bad Research Still Costs Good Money

I have my opinions about which academic research is worth funding with public money and which isn’t. I also understand if you couldn’t care less about what I think. But I expect we’ll all share similar feelings about research that’s actually been retracted by the academic journals where it was published.

Globally, millions of academic papers are published each year. Many - perhaps most - were funded by universities, charitable organizations, or governments. It’s estimated that hundreds of thousands of those papers contain serious errors, irreproducible results, or straight-up plagiarized or false content.

Not only are those papers useless, but they clog up the system and slow down the real business of science. Keeping up with the serious literature coming out in your field is hard enough, but when genuine breakthroughs are buried under thick layers of trash, there’s no hope.

Society doesn’t need those papers and taxpayers shouldn’t have to pay for their creation. The trick, however, is figuring out how to identify likely trash before we approve a grant proposal.

I just discovered a fantastic tool that can help. The good people behind the Retraction Watch site also provide a large dataset currently containing full descriptions and metadata for more than 60,000 retracted papers. The records include publication authors, titles, and subjects; reasons for the retractions; and any institutions with which the papers were associated.

Using that information, I can tell you that 798 of those 60,000 papers have an obvious Canadian connection. Around half of those papers were retracted in the last five years - so the dataset is still timely.

There’s no single Canadian institution that’s responsible for a disproportionate number of clunkers. The data contains papers associated with 168 Canadian university faculties and 400 hospital departments. University of Toronto overall has 26 references, University of British Columbia has 18, and McMaster and University of Ottawa both have nine. Research associated with various departments of Toronto’s Sick Children’s Hospital combined account for 27 retractions.

To be sure, just because your paper shows up on the list doesn’t mean you’ve done anything wrong. For example, while 20 of the retractions were from the Journal of Obstetrics and Gynaecology Canada, those were all pulled because they were out of date. That’s perfectly reasonable.

I focused on Canadian retractions identified by designations like Falsification (38 papers), Plagiarism (41), Results Not Reproducible (21), and Unreliable (130). It’s worth noting that some of those papers could have been flagged for more than one issue.

Of the 798 Canadian retractions, 218 were flagged for issues of serious concern. Here are the subjects that have been the heaviest targets for concerns about quality: