Since the start of the SARS-COV-2 (COVID-19) outbreak, scientists have repeatedly advocated for the use of well-structured clinical trials in testing new treatments for the disease. But what does a well-structured trial look like?

In part one of this series, we discussed how clinical trials are set up. In part two, we highlight a few key components to look for when reading about the latest research and clinical trials, because they are not created equal.

Maurizio Fava, MD, Psychiatrist-in-Chief at Massachusetts General Hospital and Director of the Division of Clinical Research of the Mass General Research Institute, stresses the “importance of well-designed studies and clinical trials, as today’s clinical research will help us improve the standard of care of the future. This is absolutely critical for conditions such as COVID-19, given the need to develop ways to both prevent it and treat it.”

Here are a few things to look for to ensure that results are as accurate as possible:

Sample size: The number of patients/participants studied

The number of people involved in a clinical trial is critical because scientists are basing the success of the treatment on how it affects the participants involved. These insights that get applied to an entire population, so ensuring they are as accurate as possible is important for everyone’s safety.

For example, a treatment that appears to work well in a sample of 20 participants may not work as well when that pool is expanded to hundreds or thousands of participants.

Keep in mind, a small sample size may also make key differences harder to spot and may not be representative of or applicable to a larger population. If a study is done on a small homogenous group with similar demographics (age, health status, ethnicity, gender, etc.), there is no telling how it could affect other demographics.

Placebo: An inactive substance given in the place of a treatment

Placebos are used when there is no existing standard of care to test a new treatment against. They are typically designed to look like the medication that is being tested but do not have a therapeutic effect.

Testing a new treatment against a placebo gives researchers something to compare their results to and helps to eliminate bias in patient-reported outcomes.

Randomization: Assigning treatments to participants at random

Randomization is the process of randomly assigning patients to either the treatment or control group without considering underlying factors such as disease state, age, weight or medical history.

For example, if all young participants receive an experimental treatment and get better, while all older participants receive standard treatment and fare worse, it would be difficult to prove the experimental treatment was the sole cause of improvement, because age could play a role.

However, if the experimental treatment was distributed to all participants at random and the health of the experimental group improved (regardless of age), it would be easier to draw more accurate conclusions.

Peer review: A process in which experts in the same field objectively review a scientific study before publication

Peer review is a vetting process that allows impartial subject matter experts who were not involved in the study to review research before it is published. It is critical to scientific discovery because it helps validate and improve the quality of research.

There are some cases where scientists will opt to publish their findings in a non-peer reviewed journal because it is faster and easier than going through the peer review process, but this also means there is potential for errors and findings may not be accepted by the broader scientific community.

Non-peer reviewed studies have become increasingly popular, so when reading about scientific findings or new study results, it is important to check where those results have been published.

Blinded studies: Studies that withhold treatment information from patients or researchers to reduce bias

There are two types of blinded studies: single-blind and double-blind.

• Single-blind: Researchers know if participants are receiving the treatment or the placebo/standard of care, but participants do not. This helps reduce participant bias by limiting the “placebo effect”—a form of unconscious bias that can sometimes lead people to feel better after believing they have been given a new treatment, even if it was an inert substance or standard treatment.
  • Mass General is currently a part of a blinded multicenter placebo-controlled randomized clinical trial evaluating hydroxychloroquine called ORCHID: Outcomes Related to COVID-19 Treated with Hydroxychloroquine among In-patients with Symptomatic Disease.

• Double-blind: Neither participants nor the researchers know who is receiving treatment. This is considered the “gold standard” in clinical trials because it helps reduce participant and researcher bias. With both groups having little information to influence their perspective, the study is likely to produce more accurate results.
  • Mass General is one of 54 recruiting sites for a phase III double-blind, placebo-controlled remdesivir trial called the Adaptive COVID-19 Treatment Trial (ACTT).

One of the first high profile clinical trials for hydroxychloroquine as a potential treatment for COVID-19 received sharp criticism from the scientific community due to several issues with its structure.

Critics were quick to point out that the sample size was small with just 42 participants, the control and treatment group participants did not appear to be randomly selected and several negative patient outcomes were excluded from the results.

Two additional studies from The Lancet and The New England Journal of Medicine were also recently retracted. Findings gathered from The Lancet study were called into question when the scientific community noticed homogenous patient data and potential issues with the statistical analyses. Researchers leading The New England Journal of Medicine study were forced to retract their study when they could not validate their supporting research.

As we move forward and learn of new findings from researchers working to uncover the mechanisms behind disease, it is important to ask questions and critically examine the supporting research before accepting new findings as facts.

For more information and updates about COVID-19 clinical trials at Mass General, please visit the website.

Save and share our clinical trials infographics on social media!

COVID-19 Research at Mass General
Researchers and clinicians at Massachusetts General Hospital Research Institute are mobilizing to develop new strategies to diagnose, treat and prevent COVID-19. Learn more.

About the Mass General Research Institute
Research at Massachusetts General Hospital is interwoven through more than 30 different departments, centers and institutes. Our research includes fundamental, lab-based science; clinical trials to test new drugs, devices and diagnostic tools; and community and population-based research to improve health outcomes across populations and eliminate disparities in care.
Support our Research