The Voice of Epidemiology


    Web EpiMonitor

► Home ► About ► News ► Job Bank Events ► Resources ► Contact

"Getting Risk Right": An Interview with Geoffrey Kabat

Seeking To Learn From Epidemiology At Its Best And Worst

This month The Epidemiology Monitor re-interviews Geoffrey Kabat, cancer epidemiologist at Albert Einstein College of Medicine, following publication of his latest book—Getting Risk Right, a thoughtful examination of the scientific process involved in identifying and assessing health risks. The Monitor interviewed Kabat earlier in 2009 when he published his first book entitled “Hyping Health Risks”.  (See )

 Kabat has long been concerned with the challenges facing epidemiologists in doing solid research and having these results represented accurately in the public domain.  Beginning with the basic question “Why do things that are unlikely to harm us get the most attention?”, Kabat makes use of detailed case studies to explore the factors that contribute to epidemiology both at its best and its worst. The public, the media, and the scientists all appear to contribute to the problem Kabat highlights. His analysis should be of interest to those epidemiologists in the public health community hoping to contribute to meaningful scientific advances. 

EM: The over-hyping of health risks is something you have been studying closely for some time and explored in your previous book Hyping Health Risks. What specifically motivated you to write this new book and how does it pick up where the last one left off?

GK: The first book, Hyping Health Risks, took a critical look at 4 prominent environmental exposures that received a great deal of attention in the 1980s and 90s What I tried to show, and explain to myself, was how in each case a scientific question in the area of public health could get distorted and inflated as a result of limited or flawed epidemiologic studies combined with what was made of published results by the media, advocacy groups, regulators, and scientists themselves.

As I was finishing the first book, there were other questions, like cell phones and particulate air pollution, that caught my interest.  And my editor suggested other topics, such as BPA. But my strongest feeling, as I thought about another book, was that I didn’t want to repeat myself.  To spend years writing a new book, I had to find an animating idea that felt new and exciting.

As I grappled with new topics, it came to me that I wanted to contrast instances where risks were hyped with examples of what science at its best can achieve in the area of public health. And the HPV story came to mind because I knew people at Einstein who had worked on HPV since the early 1980s and because I noticed that epidemiologists got a gleam in their eye when we talked about this success story.  What struck me, above all, is that the general public hears little about the process that led to the development of a vaccine – how long it took and how many twists and turns there were in the path leading to the identification of the specific types of HPV that cause cancer and then to the development of a vaccine -- and that achievements like this tend to be taken for granted.  On the other hand, there is enormous attention rooted in fear that is directed at potential risks that often turn out to be of little or no consequence.  So that idea – of the contrast between instances where scientific research relating to health risks gets enormous attention but fails to uncover important new knowledge or make progress, and those where unimaginable progress is made – provided the central tension of the book.  These represent two extremes.  Most research lies somewhere in the middle.  But I felt that we could learn something by contrasting the two extremes.

EM:  You devote roughly the first third of the book to discussing issues of bias and methodologic and disciplinary pitfalls.  Can you summarize the thrust of this introductory section?

GK:  The first third of the book lays the groundwork for the remainder of the book – the case studies that are the heart of the book.  I start with the question, “Why do things that are unlikely to harm us get the most attention?” and refer to the large number of contested issues involving things that might be adversely affecting our health, including vaccines, GMOs, pesticides and other chemicals in the environment, cell phones, salt, obesity, smokeless tobacco, e-cigarettes, “fracking,” etc.  But rarely does the quality of the studies enter into the public discussion. You would never know from the media that there is a lively discussion going on the quality of scientific research in the biomedical field.  Specifically, there is increasing recognition within the scientific community of what has been referred to as a crisis characterized by fierce competition for funding and professional advancement, a lack of reproducibility of published papers, and a lack of transparency.

EM: What was your main goal for this introductory section of the book? 

GK: My goal in the first 3 chapters was to provide a framework for understanding the many factors that can influence published findings and how they get reported to the public.  I cover methodological biases inherent in observational studies, as epitomized by the work of John Ioannidis and colleagues, cognitive biases such as those described by Daniel Kahneman, professional and political biases, and “bandwagon effects.” These different biases can interact and reinforce each other. My intention in this section was to provide a description of the landscape in which research is conducted and the kinds of pitfalls it is subject to.

EM: Do you feel these pitfalls apply equally across all areas of public health research?

GK: No, a crucial point, which rarely gets attention, is that all areas are not equal when it comes to the reproducibility of findings.  This is shown nicely in a 2012 analysis by Tsilidis et al. that showed that among studies using biomarkers to assess associations with cancer, infectious agents (HPV, H pylori, HBV) had robust findings.  In contrast, studies examining IGF/insulin and markers of inflammation had considerably lower reproducibility.  Finally, studies of diet and environmental exposures had very poor reproducibility. 

EM: Having set the stage, can you encapsulate the contrast between your two sets of cases studies?

GK: In the first two case studies I retell the story of roughly twenty years of research devoted to the issues of cell phones and brain cancer, and the possible health effects of exposure to “endocrine-disrupting chemicals” in the environment.  In my view, both questions have their origin in dramatic findings that galvanized the attention of scientists but were misleading.

EM: What specifically were the dramatic findings that led scientists down the wrong path for so many years?

GK: The cell phone question arose due to a man in St. Petersburg, Florida, who, after his wife died from brain cancer, brought a lawsuit against a wireless company and went on Larry King Live.  The endocrine disruption hypothesis came about in large part in the early 1990s due to three observations: incidents in which exposure to industrial runoff affected the sex of alligators and other wildlife; the DES experience in the middle of the last century; and the alleged decline in sperm counts.  None of these findings turned out to be relevant to the general population, as I explain in detail.  But they led to a line of hundreds of research studies, which sometimes produced “suggestive” results and kept the bandwagon going.

EM: What were the main factors that contributed to the propagation and continued study of these misleading hypotheses?

GK: Importantly, the results of studies generated concern in the public and led to regulatory attention.  I think one must acknowledge that the fact that these two issues were so much in the public eye and caused so much concern helped to perpetuate a line of research that has failed to produce solid evidence in favor of either hypothesis.  Being in the spotlight helped to keep what was a weak and poorly-justified hypothesis alive and consume scarce funding. 

For me, when I had a back-and-forth with the eminent expert on male reproductive function, Richard Sharpe, who is one of the originators of the endocrine disruption hypothesis, he put the issue in a way that sheds a glaring light on how a field can go wrong.  “In retrospect, I consider that circumstances helped me because I ended up disproving my own hypothesis/ideas early on in the ED saga.  Plus, I was lucky that the question that drove me was ‘what causes these disorders,’ not ‘how do EDCs cause these disorders?’  Such a simple difference, but it takes your thought processes in a very different direction.”

EM: Were there clues at the time that the original hypotheses were poorly justified? Would you suggest that these situations could be avoided if hypotheses are more solidly justified before being pursued?

GK: That’s an interesting question.  I think, to some extent, the degree of distortion could have been reduced.  Scientists are supposed to be skeptical and to be critical of the evidence, and, I think that regarding cell phones, and the earlier question of EMF, there was a tendency to have a narrow focus on weak epidemiologic studies and difficult-to-interpret in vitro studies.  What I think should have gotten more attention was the nature of the type of energy involved (i.e., microwaves and extremely low-frequency electromagnetic fields) and the plausibility that these types of very weak energy could be inducing biological effects.  I’m not saying that this consideration should have been determinative, but it should have been taken into account. Certainly, as time passed and more robust studies were done (particularly, involving whole animals exposed to radiofrequency emissions), there was a tendency for this strong null evidence to not receive the weight it deserved and to latch on to weak findings from a minority of epidemiologic studies.  So, to an extent, yes we could have been more skeptical from the outset, but, of course, the picture becomes clearer as more high-quality studies are done.  Similar considerations could have helped put the endocrine disruption hypothesis in a critical perspective.

EM: And what about the second set of case studies?

In contrast, the second set of case studies tells of work that was carried out over decades to understand 1) a mysterious disease involving irreversible kidney damage in the Balkans and 2) the etiology of cervical cancer.  Both questions were difficult and required considering multiple hypotheses, excluding explanations that did not fit with the evidence, confirming findings, and refining one’s hypothesis. Over time, there were false leads, methodological and technical obstacles that had to be overcome, and disputes between different disciplines. But over time, scientists in different parts of the world collaborated and overcame obstacles and confirmed the links in a chain of causation.  In both cases the work led to new knowledge but also to undreamed of consequences for public health. 

EM: What were those consequences?

GK: In the first case, we now know that an herb used in traditional medicine in major cultures going back two thousand years (Aristolochia) causes irreparable kidney damage and cancer of the upper urothelial tract, and the mechanism of cancer induction is now known (i.e., it is a highly specific signature mutation in TP53).  In the second case, work over more than 30 years has led to the development of vaccines that protect against HPV infection and have the potential to virtually eliminate cervical cancer, with over half a million new cases and over a quarter of a million deaths each year, mostly in countries in south Asia and Africa. 

EM: So what do you believe is the main contrast between to the two sets of studies? 

GK: The stories highlight how science that tackles an important question typically only makes progress by dint of painstaking work by different groups of scientists over time. As Harald zur Hausen has said, there was “no eureka moment.” Furthermore, because forging the links in the chain is painstaking and unglamorous, it is not newsworthy.  Rather than the reports of the latest threat or breakthrough, we should give greater attention to the hard work of science that, if it pursues an important problem, can make life-changing advances. 

EM: It seems a key distinction between these sets of case studies is whether or not the media got involved early on and played a role in influencing the research before the scientific process had enough time to play out and reach solid conclusions. Would you say the media or the scientists shoulder more of the blame in these situations?

G.K.  The media is the media.  What sells is what is unexpected, startling, and novel.  That is not going to change.  There is actually a good deal of high-quality reporting, if you know where to look for it, but that is not going to reach the vast majority of the population.  So, I don’t think one can have great expectations about the mass media changing.  As a scientist, I’m more concerned about the quality of scientific studies and the seeking out of media coverage for results that really have very little claim on the public’s attention. There is a pretense that the public needs to know about studies, but often the results really are so uncertain that one has to question what use they are to anyone.  So, I come down on the side of feeling that we need much higher standards for what gets published and to stop utilizing the public to boost the stature of our work.

EM: Today more than ever, scientists are under tremendous pressure to publish or perish. Do you think systemic reforms are needed within academia in order to balance such a desire for higher standards in publishing with the increasingly competitive modern academic environment?

GK:  I definitely think that systematic reforms are in order.  These issues have been widely discussed, for example, in an informal survey of scientists conducted by Vox , and, most recently in a paper entitled “A manifesto for reproducible science.”

(The Epidemiology Monitor covered the results of the survey mentioned above in detail in our September 2016 issue:

EM: What is the key lesson that can be taken from these case studies about how to best investigate and report on scientific results?

GK: Having in mind models of what science can achieve at its best can provide a standard by which to judge the extravagant claims based on flimsy evidence, which get so much attention.

EM: How does having a standard of what science can achieve actually help us to determine that some claims are extravagant and others are reasonable to pursue? After all, isn’t determining what is flimsy evidence from what is promising evidence the real challenge here?

That is not always easy to determine in the early days of research findings.

GK:  You are right, in the sense that the case studies that I recount only achieve maximum clarity in retrospect.  In the 1970s virologists were dismissive of Harald zur Hausen’s hypothesis that papilloma viruses might be the cause of cervical cancer.  And Richard Sharpe put forward the idea in the early 1990s that “living in a sea of estrogens” might explain abnormal reproductive development.  All one can do is to keep in mind alternative hypotheses and not develop tunnel vision, blocking out explanations that don’t fit with one’s hypothesis.  As I emphasize in the book, quoting the biophysicist John Platt, keeping in mind alternative hypotheses is the best way to protect against selecting data that appear to support a favored hypothesis.

EM: You describe a litany of factors contributing to misrepresentation of health risks. Is there any one factor you believe is most central to the problem? If not one, which do you feel are the most detrimental to promoting good science?

GK: Two things appear to me to be of paramount importance.  First, we have to avoid becoming wedded to a particular hypothesis, even if it is in vogue and provides a source of funding.  Once we block out alternative and possibly more promising hypotheses, we become prisoners of confirmation bias.  Second, the politicization of science is a serious danger. By politicization, I mean allowing an ideological stance or policy considerations to influence one’s interpretation of the evidence on a particular question. We need to continuously strive to distinguish good -- that is, reproducible -- science from politics and from policy.  Jeremy Berg, the new editor-in-chief of Science magazine, has recently made this point.             ■

Reader Comments:
Have a thought or comment on this story ?  Fill out the information below and we'll post it on this page once it's been reviewed by our editors.

  Name:        Phone:   



      ©  2011 The Epidemiology Monitor

Privacy  Terms of Use  Sitemap

Digital Smart Tools, LLC