Letter: Randomized Controlled Studies


Jacob Schor, ND

I read the article by Warnock and Bramwell in your November 4 e-newsletter with great interest and have been pondering their conclusions.

Of late I have developed the habit of watching an imaginary internal meter inside my head as I read that I imagine measures disturbances in my mental equanimity.  (Some would call this a BS meter, but I would prefer to call in a plausibility gauge.) Physically my left eyebrow seems to go up a bit and my eyelid squint when the meter registers.  While still alive, my mother told me that

I had inherited that facial tell from her father. If and when this usually steady meter is disturbed, I’ve learned to pause and pay closer attention. Sometimes I notice that my head does a slight rotation to the left and tilts slightly toward that shoulder.  In Bramwell and Warnock’s article  the meter was triggered by their supposition that a minimum of a thousand participants were required in order to offset potential bias in randomized controlled studies and that smaller groups were ineffectual.  Such requirements would obviously invalidate findings from RCTs that didn’t meet those requirements of size. 

When the meter goes off, I’ve learned to track the citations back to see where they originated; in this case this idea is from a 2017 paper in the Journal of Clinical Epidemiology by Tri-Long Nguyen et al.1 A few moments of further investigation, though, reveal a second paper on this subject, one written by Stephen Senn that appeared in the same journal in 2022.2  In his paper, Senn goes through the mathematics behind Nguyen’s calculations and points out the reasons why Nguyen’s claims prove to be misleading.

Disruptive ideas that upset the status quo have an innate attraction that arouses both our curiosity and our desire to spread them.  This is why gossip is so dangerous in a community, why false news stories have become so dangerous in national and global politics, and why we need to read carefully before spreading erroneous ideas in medicine.

The findings of Soroush Vosoughi’s MIT study on fake news that appeared in Science in 2018 are still true.3

“A false story is much more likely to go viral than a real story…. A false story reaches 1,500 people six times quicker, on average, than a true story does. And while false stories outperform the truth on every subject—including business, terrorism and war, science and technology, and entertainment—fake news about politics regularly does best.”4

When repackaging and then spreading information we gleam from the science journals about the practice of medicine, we should be cautious that we don’t cherry pick the information we deliver too carefully.  When there is competing information that would in effect nullify earlier findings, that information should be mentioned.

This obviously isn’t a new problem, but it seems to be amplified with each advance in communication technology. It seems that the faster we can transmit information to increasingly wider audiences of recipients, the less reliable that information may be.

Obviously, this isn’t a new problem as evidenced by these lines from Jonathan Swift:

“Falsehood flies, and truth comes limping after it, so that when men come to be undeceived, it is too late; the jest is over, and the tale hath had its effect: like a man, who hath thought of a good repartee when the discourse is changed, or the company parted; or like a physician, who hath found out an infallible medicine, after the patient is dead.”
                                            Jonathan Swift, The Examiner. No. XIV (Thursday, November 9th, 1710

References

  1. Nguyen TL, Collins GS, Lamy A, Devereaux PJ, Daurès JP, Landais P, Le Manach Y. Simple randomization did not protect against bias in smaller trials. J Clin Epidemiol. 2017 Apr;84:105-113.).
  2. Senn S. Empirical studies of balance do not justify a requirement for 1,000 patients per trial. J Clin Epidemiol. 2022 Aug;148:184-188. https://www.jclinepi.com/article/S0895-4356(22)00049-X/fulltext
  3. Vosoughi S, Roy D, Aral S. The spread of true and false news online. Science. 2018 Mar 9;359(6380):1146-1151. doi: 10.1126/science.aap9559. PMID: 29590045.
  4. Robinson Meyer. The Grim Conclusions of the Largest-Ever Study of Fake News. The Atlantic. March 8, 2018. https://www.theatlantic.com/technology/archive/2018/03/largest-study-ever-fake-news-mit-twitter/555104/

Published December 2, 2023

About the Author

Jacob Schor, ND, now retired, had a general practice with a focus on naturopathic oncology in Denver, Colorado. He served as Abstract & Commentary Editor for the Natural Medicine Journal for several years (https://www.naturalmedicinejournal.com/) and posts blog articles on natural therapies,  nutrition, and cancer (https://drjacobschor.wordpress.com/). He is a board member of CoAND and past president of OncANP, and someone who is happier outdoors than inside.