At this time, the primary way we are asking for people to support RIAT is by spreading awareness and information about the initiative. Few restorations of the trial literature have been done to date. The aim of the RIAT Support Center is to facilitate many more restorations by providing funding and hands-on support to researchers. Here are some ways you can help spread RIAT-awareness:
- Share news and information about RIAT through your networks (i.e., editorial organizations, academic institutions, clinical societies)
- Highlight RIAT – we’d appreciate mention in your newsletter, on your discussion lists
Additionally, we are always looking for feedback and help improving our web interface. If you have technical skills or wisdom you are willing to impart on us, please get in touch, support@restoringtrials.org.
RIAT: Restoring Invisible and Abandoned Trials
RIAT is a scientific initiative and as such has no membership. We encourage everyone to get involved in any way possible! If you have specific ideas or would like to get involved in a particular aspect of RIAT, please contact us at support@restoringtrials.org
Randomized controlled trials are known as medicine’s “gold standard” for reliable evidence. Overall, however, they suffer from substantial failures in reporting. For example: not all trials conducted are published and many that are published are inaccurately reported. This can cause research findings to look more positive than they actually are and mislead readers about the true efficacy/effectiveness and safety of a particular treatment or test.
We consider “clinical trial data” to refer to the totality of recorded information related to a clinical trial. Therefore this definition includes not only all datasets from trials, but also the many pre-trial and post-trial documents that are created in the course of designing, running, and reporting a trial (such as the trial protocol, statistical analysis plan, blank and completed case report forms, randomization lists, clinical study report, trial publication, trial registry entry, and so on). Even meeting minutes, marketing assessments, and email correspondence about trials are types of “clinical trial data,” and may be helpful or even necessary to fully understand a trial.
There are several ways one can attempt to access clinical trial data for a published study, depending on the type of study and what entity holds the data, but no certainty that one will be able to access all or part of the data. There are four broad source categories: investigators, sponsors, regulators and to some extent, public repositories. If a trial is published, asking for data from the authors of the trial publication is an easy first step (but be prepared that this may not result in data). Some sponsors maintain their own register (e.g. GSK and Merck), and some have created consortia to create a single entry portal for industry-sponsored trials. The latter require protocols and requestors go through a screening process before being granted access to data. All have limitations on data release. Regulators, chiefly EMA, require requests for documents of compounds regulated prior to January 1, 2015 but EMA is prospectively making more recent data available through its Clinical Data Publication website. FDA and Health Canada are slowly falling into line but FDA medical, statistical, and other reviews that discuss trials submitted to them, are easily accessible from Drugs@FDA.
Public repositories such as Zenodo and Dryad are freely accessible, but searching them for clinical trial data is not easy. The Drug Industry Documents archive (formerly known as DIDA) also holds a selection of clinical study reports, formerly confidential internal memoranda, and other trial data.
The procedure is nearly identical to obtaining data for a trial that has been published in a biomedical journal, with the only difference that asking the authors of the publication for data is not an option since there is no publication!
To the entity which holds the data. This might be the investigators, the sponsor, and/or a regulator. Some entities which hold data (e.g. the GSK Study Register and EMA’s Clinical Data Publication website) may not require a request; you can download documents directly from the website, including clinical study reports. This is in contrast to other entities (such as ClinicalStudyDataRequest.com) require a formal request and even a protocol. The relevant portal will tell you what to do.
If a trial is published, asking for data from the authors of the trial publication is an easy first step (but be prepared that this may not result in data). Each of the web-based data sources has its own procedure for requesting data which is described on the relevant website. The EMA has a very simple procedure which is known as Ask EMA, and consists of filling out a simple form on EMA’s website. You are assured of an answer and you do not have to specify the reason for your request nor do you have to be a citizen of the EU. For compounds on which a decision has been made since January 1, 2015, the EMA Clinical Data Publication website is a mould breaker for it does not request a request for data, only an account (the simplest of which can be created in minutes).
This term is generally used to refer to a study that had not been published in a biomedical journal. As awareness of the utility of the range of types of clinical study data has grown in the last decade, from publications to data such as clinical study reports, individual participant data, and drug approval packages, the term “unpublished study” has become more ambiguous and consequently more confusing. This is because a study may not be published in a biomedical journal, but still it may have a clinical study report that is more than 1000 pages long and there may be methods and results data from this trial recorded in trial registers. On the flip side, “published studies” may have journal publications associated with them, but study documents like the trial protocol and results reporting to a trial registry may be absent. The take-home message is that “unpublished data” and “unpublished study” are two distinct concepts.
Now, does results reporting on trial registers (like ClinicalTrials.gov) constitute publication? If we accept that the root of the word “publish” means “to make public”, a registry entry (with results), a clinical study report released by a regulator or a sponsor, a report posted on the web, a regulatory review containing sufficient information and a researcher’s report made public could fulfil the requirement for publication. However in the context of a trial, the term “report” usually means a document with an IMRAD structure, and therefore contains more than just methods and results data, but also contextual data found in the Introduction and Discussion section of a manuscript. So regulatory reviews (like a FDA medical officer report) and register entries do not fulfil the definition of report. At the RIAT Support Center, we believe that the terms “invisible” and “published in….” describe the current situation more accurately.
The most important first step is to present your concerns to the trial investigators and/or sponsor in order to determine whether or not the misreporting you have identified is indeed misreporting. If the misreporting is confirmed, the next step is to determine whether or not the trial investigators and/or sponsor intends to correct the record.
Once sufficient notice to the original investigators and sponsors has been made and you have documentation that shows or suggests the trial investigators and/or sponsor will not correct the record, the trial can be considered abandoned and you may consider restoring the trial through RIAT.
See our RIAT declaration, box 2 and “How to RIAT” page.
See our RIAT declaration, box 2 and “How to RIAT” page.
There may be more than one (a full restoration), but this is under currently review because of the resource levels required for a full restoration and the need to increase coverage of restorations.
That depends on whether what you have available is trustworthy and allows you provide a solid rationale for a restoration. You should also be explicit about what is missing (if you know).
Carrying out a RIAT is likely to involve obtaining many trial related documents including large amounts (hundreds of pages) of documents and data. Researchers are free to use whatever methods works for them.
Aligned with concept of research transparency, we encourage RIATers (i.e. those carrying out a RIAT) to make as much data related to the RIAT trial publicly available, providing it is de-identified. To do this, we recommend using the Open Science Framework (OSF). OSF is a free-of-charge cloud-based project management service that allows investigators to organize, share & edit research files much like other cloud sharing applications. It also enables researchers to make these files publicly accessible at any time, and can also be used to issue a DOI for individual documents or overall projects. Documents and projects with DOIs are identifiable by CrossRef and therefore identifiable by other researchers.
The Open Science Framework is a free, secure, online project management platform that can be used to store and share files, data, code, and protocols related to projects.
RIAT is, fundamentally, a concept for tackling bias in the literature. The RIAT Support Center, which aims to facilitate researchers who would like to carry out a RIAT, is supported by funds from the Laura and John Arnold Foundation.
Please see our “RIAT Studies” page. Note that the trials listed cover very diverse topics: a morning sickness treatment, a surgical indication for re-intervention in colorectal cancer, and an infamous antidepressant trial.
Yes, see our glossary.
CONSORT are a set of initiatives to address inadequate reporting in clinical trials. CONSORT does not include re-analysis of and restoration based on the below-water bulk of the reporting iceberg, but aims to set standards of how a randomised trial should be reported in the biomedical literature.
It is no coincidence that both RIAT and CONSORT were developed by evidence synthetisers, those who in their daily work identify the misreporting of trials. The RIAT instrument RIATAR (RIAT Audit Record) is based on the CONSORT statement, which remains the most authoritative and comprehensive checklist of all items needed for the proper reporting of randomized trials.
Restorations are based on the basis of CSRs or underlying extensive and hitherto invisible reports. The are also linked to the underlying data by a RIATAR (RIAT Audit Record) which shows which sources and data are used. RIATers must follow the original analysis plan in their restoration. If they intend carrying out additional analyses these should be clearly marked as such.
We aren’t sure at this point. But we can say that there will be multiple competitions and multiple winners over the 2017-2020 period. So stay tuned and consider signing up for our mailing list so you can be notified when we announce our next competition.
Anyone who cares about the integrity of biomedical trial literature and can show the need for restoration and the will to carry it through.
It’s hard to gain access to unpublished data and reports for trials you don’t know even exist. So the first step is learning of a trial’s existence. Published trials are often easy to find, but no so for unpublished trials. There’s no simple or quick process to find an exhaustive list of every unpublished trial conducted with a given drug, but with a little effort you may be able to at least begin identifying some unpublished trials. We suggest three approaches:
- Regulatory reports: Look for the list/table of studies that appear in easily accessible regulatory reports. The best sources are FDA medical and statistical officer reports and EMA EPARs. FDA medical and statistical officer reports often contain a “Table of Clinical Studies” and EPARs generally discuss trials in the section of the EPAR on “Pharmacological Properties” – but search around these documents. Find FDA reports via the Drugs@FDA database and EPARs can be found on EMA’s website. We have some background info about all of these resources on our Regulatory Resources and glossary pages. In particular, see glossary entries on medical officer reviews, statistical officer reviews, EPARs, and SmPCs for a sense of what each of these documents look like before you jump in.
- Trial registries: Search trial registers such as ClinicalTrials.gov or ICTRP. Trial registers were established in part with the explicit purpose of trying to decrease the number of stealth trials i.e. trials for which one doesn’t even know of their existence. In addition, some sponsors maintain their own registers (e.g. Merck and GSK) or are members of consortiums (e.g. ClinicalStudyDataRequest.com) where clinical trial information is posted. Searching these sources by drug may help identify some unpublished trials.
- Other: If the medical product you’re interested in has been involved in litigation, you may also want to check the Drug Industry Documents Archive to see what kinds of documents might be in that repository.