Big science: incentives

With the stagnation of public funding sources and the competition for academic or industry positions tougher than ever before, scientists, especially those vying for precious tenure-track positions, are under substantial pressure to make a name for themselves.

But at the same time, as the number of researchers continues to increase, it has become harder to evaluate them and determine who deserves grant money or a tenure position. Success in science has been boiled down into simple metrics, for example: the number of papers published, the impact factor of the journal publishing the paper and the number of citations the paper has accumulated. In an interview with ScienceNews, psychologist Brian Nosek claims that “[r]eal evaluation of scientific quality is as hard as doing the science in the first place…So, just like everyone else, scientists use heuristics to evaluate each other's work when they don't have time to dig into it for a complete evaluation.” As a result, the system in place heavily incentivizes both a constant stream of publications and splashy, novel results, which each have negative effects on good science.

The “publish-or-perish” mantra is most prevalent for assistant professors who are working towards attaining a tenured position. In the limited time they have to prove themselves to their colleagues, they must be able to display proficiency in developing a successful research program, while simultaneously training and teaching students.

Anyone who has dabbled in research, especially in the natural sciences, can attest that coming up with a working project, let alone a groundbreaking discovery, from scratch within just a year or two is not an easy task. In fields like chemistry or biology, it’s easier and less risky to stretch out pre-established ideas instead of coming up with new ideas that have low probabilities of becoming successful. Often, new professors pursue research that’s simply a direct corollary of existing discoveries and doesn’t lead to any particularly new studies. Thus, placing value on the number of publications can lead to wastes of time, energy and money.

More imperative is the pressure on making these publications seem better than they actually are. There’s a glaring conflict of interest in the scientific world, in that the fundamental responsibility of a scientist is to execute a study and judge the results to determine whether his/her hypothesis is valid. But when the scientist so badly wants that result to be positive, it’s very likely that there will be inherent bias, even if it’s subconscious. Whether intentional or not, bias plays a significant role in science, from its initial designs to analysis of the results. A great deal of literature has been written on the subject of scientific bias and how to avoid it, but nevertheless a meta-study published in The Lancet estimates that 85 percent of the 240 billion dollars invested in life sciences research was wasted on inefficiencies and bad science.

Fortunately, these problems can be repaired, although it will take considerable effort to do so. The root of the problem is the incentivization of positive results. The mainstream media also tends to fixate on sensational topics when it comes to reporting scientific discoveries, often misunderstanding and describing misleading claims. To a scientist, negative results can be just as helpful as, if not more so than positive results. Although it may be difficult to determine which negative results are actually beneficial, there are journals, like the “Journal of Negative Results in Biomedicine,” that are making strides towards this publishing these kinds of conclusions. Unfortunately, these journals have encountered limited success because of the negative stigma associated with publishing in these journals. Professors who submit articles to journals that focus on negative results are often looked down upon, and their future work may not be seriously considered by their peers.

Furthermore, focusing on the robustness of research methods and design would help encourage methodical thinking and deter the sloppy science that’s riddled with false positives. The thought process that fuels great discoveries, which is perhaps most valuable to those reading about them, tends to be obscured in our current outcome-based system.

Junu Bae is a graduate student in the chemistry department. His column, "interested in everything," runs on alternate Fridays.