Evidence Based Medicine and the Replication Crisis: A View From Outside

Evidence Based Medicine and the Replication Crisis: A View From Outside

When I was a collection development librarian in a previous life, the university at which I was employed was opening a medical school. I spent many hours talking to the new medical librarian about how to build collections that would support the new college, and since I had never worked with a medical school before I was learning something new every day. Thankfully, the medical librarian was experienced and we formed good connections with the new faculty and administration to be sure that we were building the collections they needed on opening day.

It was at this time that I learned about something called Evidence Based Medicine (EBM)--or medical practice that would be based on the best and most current published clinical evidence. This was to be a foundation of what the new college taught. I learned about resources such as EBSCO’s DynaMed, described thus:

DynaMed delivers the most current and accurate, evidence-based content for practicing clinicians. Our multidisciplinary team systematically and objectively surveys emergent scientific literature and clinical practice guidelines. Relevant information is captured, appraised, and synthesized rapidly into DynaMed’s clinical content…”

It always seemed reassuring that EBM was based on the latest research. It seemed desirable that physicians would not be working with what they learned in medical school twenty or thirty years before. After all, published medicine based on clinical trials is the solid foundation upon which new research and clinical practice is built, isn’t it? What’s not to like?

Now, I have since learned that there are profound disagreements among medical researchers and practitioners concerning EBM. But my purpose in bringing up EBM relates more to the replication crisis that is currently in the news. As a non-specialist with a humanities education, I have to say that the replication crisis in medical publishing makes me think differently about EBM, as well. If the purpose of EBM is to equip physicians with the latest research so that they can make the best clinical decisions, doesn’t it matter if the latest research can be shown to be pretty uneven, at best? Doesn’t it compromise the decision making of physicians by providing unsupported or misleading information?

Obviously, I don’t say any of this as a participant in medical controversy. It’s more that, as a participant in the world of scholarly communication, the news that raises doubts about the reliability of medical research also gives me pause as a patient whose physician may be using an EBM-based decision tool! After all, if one of the criticisms of EBM is that it renders the clinical experience of the individual physician secondary to the authority of the published literature, perhaps the replication crisis says something in favor of the experience of the individual physician, after all? Or maybe, if medical publishing can be cleaned up, it will make EBM more attractive?

At any rate, I dare not venture onto that specialized ground! All I can say is: medical publishing and medical researchers need to address the shortcomings that are being exposed in this replication crisis--not as an exercise in damage limitation, but with real improvements that allow people to regain confidence in the findings of a very important field of research and practice.

By Matthew Ismail

Featured Image Credits: Chemistry Lab, July 8, 2009   | © Courtesy of canyon289/Flickr.

Please sign in or register for FREE

If you are a registered user on Open Research Community, please sign in

Go to the profile of Pablo Markin
over 1 year ago

The evidence-based science discussion highlights other possible factors that have cumulatively and likely led to the emergence of Open Access. At the end of the day, among the major issues that scholarly journal publishing in Open Access faces is the soundness of its gatekeeping mechanisms. While Open Access models cannot necessarily resolve the replication or reproducibility crises, they do likely require a rethinking of the roles that libraries and librarians play in the scholarly publishing ecology or multiple ones, such as in specific subject areas. Incidentally, the replication crisis also invites a reconsideration of collection building practices and their larger-scale implications. In an augmented fashion, the same applies to Open Access collections, data sets and publications that can be freely explored, mined for insights and built upon. In this respect, in so far as the control mechanisms of traditional closed-access science take time to accomplish their job, are exposed to small-world effects, such as a narrow circle of relevant specialist reviewers, and entail attendant cost structures, by being open to changes, variation and innovation in their practices Open Science models also bring with them a significant increase in the speed with which scientific discourse can move forward, such as at the pace of blog or Twitter posts. Likewise, since the retraction performance of Open Access preprint servers or publication venues has largely been found to be comparable with that of closed-access scholarly journals, one could argue that ongoing, new and mutating crises across scientific discourses have also possibly powered a phase transition to Open Access as an assemblage of approaches to scholarly publishing amenable to the revision of the basic assumptions of various research fields in practical terms.