The Scientific Graveyard

I think it's time for a new publication, one in which we can document the things that don't work! In the "good old days" negative results were reported along with positive ones, because they do contain important information that other researchers need. But now, in this day of cut-throat competition for funding and prestige, the negative results get swept under the carpet, unmentioned, becoming the rocks upon which research careers run aground.

The other day in journal group this came up. People have been trying for years to develop and tune force fields for simulations, particularly for protein folding (ick, for the record), and we know by word-of-mouth that X tried something that didn't really work, and Y was talking about trying some sort of approach like W. But we don't know any more, and the lack of publications suggests that they weren't successful. It would be very useful to know what attempts were made, and what they found when it didn't work. But, whether because they don't get submitted or don't get accepted, papers on these things aren't appearing in top journals (or any journals, really; sometimes they're buried in a thesis somewhere).

This is a general principle. If it didn't work, perhaps there is something in the data that suggests another approach. But wait, nobody knows. That graduate student's work? It didn't really exist. Nothing to see here, move along. And, hey let's not put up a road-out-ahead sign, so that more time will be wasted. Like the cat that knocked over the vase, let's just walk away like nothing happened.

My theory for why this is the case is that the funding is so limited, and so based on prestige, that nobody will discuss it. I think this is extremely deterimental to the field (and to all the fields that this occurs in), because it's wasteful and because it moves away from a fundamental tenet of science, that a negative result is am important result.

So, in this age of blogs and ready access to information, should we have such a respository? A graveyard for the projects that went terribly wrong. I think it's a great idea.