Are Some Researchers Fabricating Data?

On May 19, 2015, an “expression of concern” appeared in the Journal of the American Medical Association (JAMA) regarding a study published in that journal in 2005. In the 2005 study, conducted by a group of Japanese researchers (Sato et al.), supplementation with 5 mg per day of folic acid and 1500 µg per day of vitamin B12 for 2 years reduced the incidence of hip fractures by 78% (p < 0.001), compared with a placebo, in elderly patients with high plasma homocysteine levels and a history of a stroke.1 The “expression of concern” was an announcement that various issues had been raised about the conduct, integrity, and scientific validity of the 2005 study. After communicating these concerns to Sato and evaluating his response, the editor in chief of JAMA contacted administrative officials at Sato’s institution and requested that they conduct an investigation to evaluate the scientific integrity of the research and the validity of the reported results.2

I have frequently cited this study in my lectures and writings, because the results were so impressive. It is rare to find such a dramatic reduction in fracture incidence in clinical trials. I am aware of only 4 other studies in which the reduction in fracture incidence was so profound (ranging from 73% to 90%). In 3 of those studies, patients with Alzheimer’s disease, Parkinson’s disease, or a history of a stroke were treated with sunlight exposure.3–5 In the other study, women with Parkinson’s disease were given 45 mg per day of vitamin K2 (menaquionine-4).6 Interestingly, all 4 of these studies were conducted by Sato et al.

My eyebrows had previously been raised by the awareness that Sato was consistently getting better results than other osteoporosis researchers. In particular, other researchers were unable to confirm a protective effect of homocysteine-lowering B-vitamin therapy against fractures in a group of patients with hyperhomocysteinemia and cerebrovascular disease.7 However, there was no other obvious reason to question the integrity of Sato’s research. Now that JAMA has raised concerns, I am wondering whether all of Sato’s research should be considered unreliable until evidence appears to the contrary.

In the past, I have publicly questioned the validity of studies reported by two different research groups; one from India and the other from Iran. In one case, I looked more closely at certain studies because it seemed implausible that a relatively small research group could have conducted numerous large-scale randomized controlled trials in a relatively short period of time. In the other case, I looked more closely at results that seemed too good to be true, after the Lancet had raised concerns about scientific misconduct in relation to other research conducted by the same investigator.8 In both instances, a closer examination of the studies revealed that some of the data were mathematically impossible or biologically inconceivable. One of the research groups that I questioned has now had six different studies retracted by medical journals. I am currently keeping my eye on a few other research groups whose sample sizes seem too large for their resources or whose results seem “too perfect.”

How should writers and teachers deal with published research when its validity is in question? Should one just ignore it, or would it be better to present the research while voicing concerns about its reliability? And why do some researchers apparently fabricate data? The most likely reasons are that having research published may increase one’s prestige in the scientific community and increase the likelihood of receiving grant money. In addition, researchers who have a financial stake in a particular product would benefit from claims that the product is beneficial.

In recent years, there has been a proliferation of open-access journals, which are supported by publication fees paid by authors. A recent study found that the peer-review process in these journals is sometimes quite lax, which means that it is easier than ever to publish questionable research. It is therefore incumbent upon us to remain vigilant in our review of research, and to maintain a healthy degree of skepticism when the situation warrants.9

Alan R. Gaby, MD

Notes

1. Sato Y et al. Effect of folate and mecobalamin on hip fractures in patients with stroke: a randomized controlled trial. JAMA. 2005;293:1082–1088.

2. Bauchner H, Fontanarosa PB. Expression of Concern: Sato et al. Effect of folate and mecobalamin on hip fractures in patients with stroke: a randomized controlled trial. JAMA. 2005;293(9):1082–1088. JAMA. 2015;313:1914.

3. Sato Y et al. Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in hospitalized, elderly women with Alzheimer’s disease: a randomized controlled trial. J Bone Miner Res. 2005;20:1327–1333.

4. Sato Y et al. Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in Parkinson’s disease. Parkinsonism Relat Disord. 2011;17:22–26.

5. Sato Y et al. Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in stroke patients. Neurology. 2003;61:338–342.

6. Sato Y et al. Amelioration of osteoporosis by menatetrenone in elderly female Parkinson’s disease patients with vitamin D deficiency. Bone 2002;31:114–118.

7. Gommans J. The effect of homocysteine-lowering with B-vitamins on osteoporotic fractures in patients with cerebrovascular disease: substudy of VITATOPS, a randomised placebo-controlled trial. BMC Geriatr. 2013;13:88.

8. Horton R. Expression of concern: Indo-Mediterranean Diet Heart Study. Lancet. 2005;366:354–356.

9. Bohannan J. Who’s afraid of peer review? Science. 2013;342:60–65.