Scientists Simplifying Science

The devil is in the variables: correlation, causation and experimental controls

SHARE THIS

On Jan 31, 2017, the New York Times reported the solving of a mysterious illness in India- the yearly seasonal deaths of children around areas of Muzaffarpur that abounded in litchi orchards. The report reads like a great epidemiological investigation piece. The supporting study was published in Lancet Global Health on Jan30, 2017 titled “Association of acute toxic encephalopathy with litchi consumption in an outbreak in Muzaffarpur, India, 2014: a case-control study”. In authors words, the key findings that led them to this conclusion were the results of their case-controlled studies where they identified metabolites of hypoglycin A or methylenecyclopropylglycine (MCPG) in patient samples BUT NOT CONTROLS. They did similar and SIMULTANEOUS INVESTIGATIONS to rule out viruses, heavy metals, and pesticides.

 

The study makes it the first complete investigation to identify the toxin as one of the factors required to setting off the illness. As pointed out by the authors, while discussing the results in their publication, they caution, “Although litchi fruits are ubiquitous in the orchards surrounding the villages in rural Muzaffarpur, typically only one child in an entire village develops this acute illness. The synergistic combination of litchi consumption, a missed evening meal, and other potential factors such as poor nutritional status, eating a greater number of litchis, and as yet unidentified genetic differences might be needed to produce this illness”. Clearly, there is more to learn about the mysterious illness although the main culprit has been identified. It is important to see that the authors are not rushing to judgment and concluding that the mystery is solved and the case rests.

 

As the saying goes, “the devil is the details”. In epidemiology, the devil is in the variables. Remember the old saying, “correlation is not causation”. This is a case in point. One variable has been associated with experimental evidence to an illness. But the results indicate that there are many other variables likely in play and more studies need to done to identify those. It is dangerous to assume that the mystery is solved and that is precisely what the authors are cautioning the readers.

 

Once a mystery is “solved”, the solution appears simple and even visible to plain sight. Such hindsight has to be applied with caution. That is a lesson many scientists would have learned from personal experience. Until one gets there, the road is paved with mines.

 

The solution to the mystery illness soon escalated into controversy. On Feb 3, 2017, The Hindu, a daily from India, reported complaints from Dr. T. Jacob John, a virologist who was earlier attached to the Christian Medical College, (CMC) Vellore. Dr. John’s contention was that the Lancet publication “quote our study but don’t honestly say what we have found. If they did that then they can’t claim originality. They have done a large case-control study but borrowed all important information connected with the illness from us,”. The questions are, did the Lancet publication borrow from Dr. John’s earlier work and whether adequate credit was given. It is a difficult question, as many times it may seem to depend on personal opinion. But careful analysis can help to dissect some of these contentions. If one looks at Dr. John’s publications, one can see several reasons why he is upset. He was very close to sealing the deal. We do not know the struggles he might have undergone to get even where he got. However, he failed to net the winning goal. My analysis is below.

 

Dr. John had published a lucid hypothesis on the illness in Current Science in 2014 that connected litchi and malnutrition with the illness. He was the first one to do so. His reasons were the correlation of phenotypes of litchi-associated illness with a similar ackee-associated illness, reported way back in the 1950’s from Jamaica. Well, he was lucky as his hunch turned out to be true. It could have turned out to be false also as he did not have the key evidence needed for his hypothesis when he published it- the presence of the toxin in the fruit. Further, as shown in Lancet, one would see that all malnourished children do not develop the disease. In that sense, Dr. John’s hypothesis is still a stretch and dangerous one too as it was ruling out other hidden variables that are still in play without adequate support. In essence, the hunch that was published as a hypothesis is an incomplete one in light of the experimental evidence from case-controlled studies published in Lancet.

 

Although Dr. John ruled out infection with Japanese Encephalitis virus  (JEV), based on other groups’ findings, he did not rule out many other infectious agents. The Lancet study tested for JEV, West Nile virus (WNV), Herpes simplex viruses 1 and 2, human herpes viruses 6 and 7, cytomegalovirus, varicella zoster virus, Epstein-Barr virus, parechovirus, adenovirus, 68 enteroviruses, and parvovirus B19. It ruled all of them out. They also tested for pesticides and heavy metals and ruled them out. More importantly, these and other studies they did were all simultaneous. Simultaneous is very important in epidemiology. Therefore, it is unfair to say that they borrowed all the important information. In fact, and as I said earlier, it looks like that more important information is yet to come that could explain why all the malnourished children in and around Muzaffarpur do not get affected by MPCG or Hypoglycin from litchi consumption.

 

In another publication entitled “Outbreaks of Hypoglycemic Encephalopathy in Muzaffarpur, India: Are These Caused by Toxins in Litchi Fruit?”, Dr. John contends that “Since the part of the fruit consumed by humans contains MCPG, and experimentally it has been shown to cause hypoglycemia in starved animals, the biological plausibility of causality by MCPG is thus confirmed.” However, it is important to note that the publication is a letter-type opinion article and a criticism of IAP members, who continued to call the illness a mystery illness. It does not present any additional data that the Lancet publication should have cited or considered. Just because a fruit contains MCPG, it need not cause illness. The Lancet publication discusses this.

 

In a Current Science publication entitled “Litchi fruit contains methylene cyclopropyl-glycine”, Dr. John’s group identifies MCPG for the first time in litchi fruits. But, it fails to associate it with the mystery illness. The experiments were all directed to the identification of MCPG in the fruit. The Lancet paper cites Dr. John’s publication where they mention, “Of specific interest was the potential role of methylenecyclopropylglycine (MCPG), a homologue of hypoglycin A, and a substance naturally found in the litchi seed and fruit known to cause hypoglycaemia in animal studies by inhibiting β-oxidation of fatty acids and gluconeogenesis”. They also cite other publications that studied hypoglycemia and toxicity of MCPG in animals.

 

In my opinion, the Lancet study can and should be considered original as it directly associates a key variable, namely the metabolites of the toxins from the fruits, with the mysterious condition, with proper controls and the simultaneous ruling out of several other variables that could have been associated with causing the illness. This goes beyond mere correlation. No doubt Dr. John on a hunch had hypothesized the cause based on a correlation. But correlation does not seal causation especially in the absence of proper controls as Dr. John himself admits in his publication “All case-children were given 10% dextrose infusion in addition to per-protocol anticonvulsants and supportive nursing care. The response to dextrose was encouraging. We are unable to compare it with any control children”. He further states, “We need to study if under-nutrition is a determinant (risk factor) or just background noise”.

 

What happened in the following days after the outcry in The Hindu national amounts to ethical misconduct. First, Current Science publishes another letter from Dr. John entitled, “Publishing on hypoglycemic encephalopathy, borrowing information without giving credit: is Current Science invisible?”, that complains about the Lancet group not giving him credit for his hypothesis. Second, without any due diligence, the Hindu national picks on the letter and rushes to judgment that “‘Litchi disease’ report: Lancet makes amends after failing to give ‘credit’.

 

First, Lancet is not making any amends after failing to give credit as they have only contacted the respective authors to find out what is going on and get to the facts. It is irresponsible on the part of the editor of a national to rush and publish such misleading articles that unnecessarily tries to mix nationalism with science, and emotions such as large funding, “large group” (Dr. John’s words) etc. Second, the step taken by the journal is routine in cases where there are complaints about not giving credit. They do not depend on newspaper items in national dailies for initiating such steps. I have personal experience myself and I am sure, the editor of the national would have found out if he discussed with a sufficient number of scientists before submitting his misleading piece for publication. It sets a dangerous trend and we should hope that the editors from the daily takes note of this and carefully reviews the material it publishes.

 

Worse, in the Current Science complaint, Dr. John misstates that an MMWR publication by the Lancet group in 2015 entitled Outbreaks of Unexplained Neurologic Illness — Muzaffarpur, India, 2013–2014, concluded that the illness is caused by Litchi consumption. His Dr. John’s own words, “the stating that the disease is acute hypoglycemic encephalopathy with putative association

with litchi as if they were the first to arrive at such a conclusion” is unethical and misleads the public. The MMWR publication does not make that conclusion. Their conclusion as stated in the publication is “Until an etiology for this illness is identified, current public health and clinical recommendations are focused on reducing mortality by ensuring families with affected children rapidly access medical attention, and health care providers promptly assess for and correct hypoglycemia”. What were the Current Science editors thinking? What were the editors of The Hindu reviewing? If there is ethical misconduct, this is one- misleading the public in a one-sided manner.

 

The Hindu article also puts words like “at a time when the Lancet authors were looking for a viral cause”. It is false. It is also unethical. If one reads the MMWR publication, one can clearly see that the authors are equivocal about the causes when they conclude their article. I have added the excerpt from their conclusion above.
Finally, it is funny that one of the authors of the Current Science article Dr. Das, says, “It is very decent of Lancet to have got in touch with us and seek our clarification on this issue. That shows Lancet is careful of its credibility, careful that any such issue is immediately sorted out. I admire Lancet . This is what we want from every journal publisher, every investigator and every paper. Caesar’s wife must be above suspicion,”. Yes. How about Current Science and The Hindu? Have they sought clarifications from the Lancet group before rushing and publishing complaints? Have they even bothered to read the controversial publications and asked for a third or a fourth opinion? The statement that “Dr. John and Dr. Das had exposed the “scientific misconduct” by the authors in a correspondence published in Current Science” in a national daily is dangerous and one-sided. It has to be condemned. It fails the ethical standards of journalism.

 

In spite of all this, I feel that the Lancet paper could have been more gracious to Dr. John’s publications. For example, the Lancet article says, “No studies implicated a direct epidemiological association between litchi consumption in affected individuals and encephalopathy”. The authors could have given Dr. John the credit for implying an epidemiological association between litchi consumption in affected individuals and encephalopathy and even removed the word “direct” from the above statement. Looks like “direct” epidemiological association” as opposed to correlation will haunt this mystery illness forever. That said, a hypothesis based on a hunch is just that and it cannot substitute for experimental evidence obtained from carefully executed research. Publishing piecemeal correlations and research without waiting to complete an investigation comes with its disadvantages. Lack and funding and support also have its disadvantages. But one cannot use that to inject moral superiority into research done by teams that operate under resource-limiting conditions. These are separate issues and should be discussed separately.

 

About the author:

Syam

Authored by Dr Syam Anand, PhD (Indian Institute of Science, IISc; Post-Doctoral research, University of Pittsburgh School of Medicine; Faculty, University of Pittsburgh School of Medicine, Founder and US Patent Agent, Mainline Intellectual Property LLC, Ardmore, Philadelphia USA). Syam has over 20 years experience in diverse areas of Science with domain knowledge in Life Sciences and Intellectual Property. Dr. Anand is also an inventor and budding entrepreneur. A rationalist, Dr. Anand enjoys science at all levels and advocates the use of scientific methods for answering all questions and solving all problems and make common people curious and interested in understanding their worlds.

https://www.linkedin.com/in/syamprasadanand

Featured image: Pixabay

Creative Commons License
This work by ClubSciWri is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

SHARE THIS

The contents of Club SciWri are the copyright of Ph.D. Career Support Group for STEM PhDs (A US Non-Profit 501(c)3, PhDCSG is an initiative of the alumni of the Indian Institute of Science, Bangalore. The primary aim of this group is to build a NETWORK among scientists, engineers, and entrepreneurs).

This work by Club SciWri is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Tags

Latest from Club SciWri