Wednesday, December 4, 2013

Half of All Trials Unpublished*

(*For certain possibly nonstandard uses of the word "unpublished")

This is an odd little study. Instead of looking at registered trials and following them through to publication, this study starts with a random sample of phase 3 and 4 drug trials that already had results posted on ClinicalTrials.gov - so in one, very obvious sense, none of the trials in this study went unpublished.

Timing and Completeness of Trial Results Posted at ClinicalTrials.gov and Published in Journals
Carolina Riveros, Agnes Dechartres, Elodie Perrodeau, Romana Haneef, Isabelle Boutron, Philippe Ravaud



But here the authors are concerned with publication in medical journals, and they were only able to locate journal articles covering about half (297/594) of trials with registered results. 

It's hard to know what to make of these results, exactly. Some of the "missing" trials may be published in the future (a possibility the authors acknowledge), some may have been rejected by one or more journals (FDAAA requires posting the results to ClinicalTrials.gov, but it certainly doesn't require journals to accept trial reports), and some may be pre-FDAAA trials that sponsors have retroactively added to ClinicalTrials.gov even though development on the drug has ceased.

It would have been helpful had the authors reported journal publication rates stratified by the year the trials completed - this would have at least given us some hints regarding the above. More than anything I still find it absolutely bizarre that in a study this small, the entire dataset is not published for review.

One potential concern is the search methodology used by the authors to match posted and published trials. If the easy routes (link to article already provided in ClinicalTrials.gov, or NCT number found in a PubMed search) failed, a manual search was performed:
The articles identified through the search had to match the corresponding trial in terms of the information registered at ClinicalTrials.gov (i.e., same objective, same sample size, same primary outcome, same location, same responsible party, same trial phase, and same sponsor) and had to present results for the primary outcome. 
So it appears that a reviewed had to score the journal article as an exact match on 8 criteria in order for the trial to be considered the same. That could easily lead to exclusion of journal articles on the basis of very insubstantial differences. The authors provide no detail on this; and again, that would be easy to verify if the study dataset was published. 

The reason I harp on this, and worry about the matching methodology, is that two of the authors of this study were also involved in a methodologically opaque and flawed study about clinical trial results posted in the JCO. In that study, as well, the authors appeared to use an incorrect methodology to identify published clinical trials. When I pointed the issues out, the corresponding author merely reiterated what was already (insufficiently) in the paper's Methodology section.

I find it strange beyond belief, and more than a little hypocritical, that researchers would use a public, taxpayer-funded database as the basis of their studies, and yet refuse to provide their data for public review. There are no technological or logistical issues preventing this kind of sharing, and there is an obvious ethical point in favor of transparency.

But if the authors are reasonably close to correct in their results, I'm not sure what to make of this study. 

The Nature article covering this study contend that
[T]he [ClinicalTrials.gov] database was never meant to replace journal publications, which often contain longer descriptions of methods and results and are the basis for big reviews of research on a given drug.
I suppose that some journal articles have better methodology sections, although this is far from universally true (and, like this study here, these methods are often quite opaquely described and don't support replication). As for results, I don't believe that's the case. In this study, the opposite was true: ClinicalTrial.gov results were generally more complete than journal results. And I have no idea why the registry wouldn't surpass journals as a more reliable and complete source of information for "big reviews".

Perhaps it is a function of my love of getting my hands dirty digging into the data, but if we are witnessing a turning point where journal articles take a distant back seat to the ClinicalTrials.gov registry, I'm enthused. ClinicalTrials.gov is public, free, and contains structured data; journal articles are expensive, unparsable, and generally written in painfully unclear language. To me, there's really no contest. 

ResearchBlogging.org Carolina Riveros, Agnes Dechartres, Elodie Perrodeau, Romana Haneef, Isabelle Boutron, & Philippe Ravaud (2013). Timing and Completeness of Trial Results Posted at ClinicalTrials.gov and Published in Journals PLoS Medicine DOI: 10.1371/journal.pmed.1001566

10 comments:

essay writing service review said...

I read this article and no doubt its full of information but honestly speaking i am least interested in medical articles and blogs etc. My sister is in medicine and she likes such.Thanks for post.

acep saepul milah said...

When you place an order Tanaman Obat Herbal Untuk Ambeien we’ll pair it with a writer who has relevant experience your topic and type of paper. You’ll get top-notch results from our experts. Plus, this experience will help you learn how proper academic writing is done. Tanaman Obat Ambeien That’s what matters the most!

acep saepul milah said...

Thanks for the information, Very useful
Manfaat kunyit Untuk Mengobati Herpes
Manfaat Lidah Buaya Untuk Mengatasi Herpes
Manfaat Bawang Putih Untuk Mengatasi Herpes

Walatra Bersih Wanita said...

Thanks for the information, Very useful
Walatra Bersih Wanita
Walatra Bersih Wanita
Walatra Bersih Wanita

acep saepul milah said...

Thanks for the information, Very useful
Cara Menyembuhkan Penyakit Hipertiroid Dengan Cepat & Alami
Inilah 7 Penyebab Hipertiroid Yang Harus Diketahui

acep saepul milah said...

We want you to enjoy your student life and not miss the fun part because of the stress of writing dissertations and essays. Therefore, we always provide a complete plagiarism report along with our final work. Walatra Propolis Brazil Asli Dan Terbaik Di Dunia

Obat Syaraf Kejepit said...

Then end users should sign-up with procedure and choose presently operating films kind listing and search for information of theaters and ebook ticket as a result of online payment making use of credit card or visa card. Visit : Obat Syaraf Kejepit Paling Ampuh

acep saepul milah said...

A more tips here procedure with two programmers possesses higher prospective for the generation of more numerous solutions to complications for 3 reasons. Aside from that, Softage largely targets devising Sophisticated solutions that ensures substantial performance.
Obat Penyakit Kuning Pada Anak
Obat Penyakit Kuning Pada Orang Dewasa
Obat Penyakit Kuning Paling Ampuh

Walatra Madu Murni said...

When you listen to people in your native language, you do not translate. You simply understand. The same has to be in English. When you learn English, you have to learn the whole sentences in context. Walatra Madu Murni | 100% Alami Dari Alam Indonesia

nabila widianti said...

Take every chance you get in life, because some things only happen once
6 Tanaman Obat Herbal Untuk Mengobati Kurap Sampai Tuntas,