Society for
Epidemiologic Research (SER) Presidential Addresses
Kuller
Presidential Address Identifies Seven Curable “Sins of Epidemiology”
Says Future of
Epidemiology Depends on Doing More Consequential Research
What can we do besides having a
better PR (public relations) firm? That’s a question asked by
Lewis Kuller in his 1996 presidential address in front one of
the largest groups of epidemiologists ever to attend a Society for
Epidemiologic Research (SER) meeting in Boston in mid-June. Noting all
the publicity about epidemiologic findings in the last year, including
a Science article highly critical of epidemiology, Kuller offered a
list of curable sins which he said might be contributing to
epidemiology’s fall from scientific grace.
The seven sins described by
Kuller, who is chairman of the Department of Epidemiology at the
University of Pittsburgh, are:
1) the sin of biological
implausibility
2) the sin of confusing
reproducibility of a measure with accuracy or validity—the “parrot
effect”
3) the sin of adjusting data
unnecessarily—the “adjustment of data syndrome”
4) the sin of failing to
determine temporal associations—the “epi phenomenon”
5) the sin of using multivariate
models to evaluate biological or causal associations
6) the sin of replicating the
same epidemiological studies over and over again with minor variations
on the theme
7) the sin of disassociating
academic epidemiology from public health
Biological
Implausibility
This occurs when the
investigator does not know or understand the biology or
pathophysiology of the disease of interest or the measures of the
independent and dependent variables. The reported results of the
study, i.e. the measurements, are non-physiological. The investigator
is unaware of the problem. This is more likely to occur when the data,
usually the independent variable, is transformed and the units of
measurement are abstruse. Investigators, reviewers, and journal
editors have no idea of the range of the true measurement.
“Parrot Effect”
Comparison of different methods
of measuring the same variable does not necessarily prove that any of
the measures are accurate. It just assumes that the correlations among
the measurements are fairly high, i.e. repeatability but not accuracy.
According to Kuller, “you can teach a parrot to say good morning. The
parrot says good morning all day long very repeatably, but the parrot
is unfortunately inaccurate most of the time.”
The Adjustment of Data
Syndrome
“I do not believe that one can
truly adjust for sex, except by a surgical procedure that is very
unpopular,” says Kuller. Also, he believes that adjustment for host
specific factors eliminates host susceptibility as an item to be
studied, and calls it “a dreadful mistake.” Furthermore, age
adjustment presumes similar biological relationships for young and old
individuals, and this is “clearly illogical.”
The Failure to Determine
Temporal Associations
According to Kuller, it is
likely that the independent variables are really due to the dependent
variables for many of the associations reported in epidemiology. The
levels of many risk factors in both cardiovascular disease, cancer,
and probably other chronic diseases are modified by the disease in its
early subclinical phases.
Multivariate Models To
Evaluate Biological or Causal Associations
According to Kuller,
epidemiologists often use multivariate models to exclude co-variates
because their p-value was less than .10 or .05 in the model. Some of
the variables that remain in the model are clearly dependent variables
of the independent variables. “There is nothing wrong with generating
a new testable model of associations,” says Kuller, “the
epidemiologist however must document the nature of the new model, how
it relates to existing biological interpretations and how the model
can be further tested in both epidemiological and laboratory
sciences...The use of complex statistical methodology is not an answer
for either good study design or biological plausibility.”
Replicating Epidemiolgic
Studies
To make his point, Dr. Kuller
asked—Are many epidemiological studies to be classified as
inconsequential because they only represent fine tuning of previous
studies with slightly different populations with little likelihood of
enhancing the science of epidemiology and the public health and
preventive medicine implications? According to Kuller, such studies
are often methodologically outstanding, the analysis is superior, and
the study is dull! In his view, the basic problem is the failure to
move from descriptive to observational studies to clinical
trials/natural experiments in order to further our understanding of
potential etiologic associations.
Schism Between Academic
Epi and Public Health
This sin is probably the most
important, according to Kuller, and involves failure on the part of
both the public health community and the epidemiology community.
According to Kuller, the public health establishment in the United
States has, to a considerable degree, lost its epidemiology base and
similarly epidemiology has lost its public health base to a
considerable degree. “It is time for academic epidemiologists to take
a very hard look at what is public health epidemiology and preventive
medicine and re-focus some of our efforts in establishing a strong
scientific basis to evolving public health and preventive medicine
programs.” The use of epidemiology to identify risk and evaluate
public health (and to develop) preventive strategies will continue to
be a “big winner,” says Kuller.
Conclusion
Dr. Kuller offered this view in
his concluding remarks. “We are very good when we study epidemics,
have well-defined and testable hypotheses that will lead to effective
public health and preventive medicine techniques that reduce morbidity
and mortality and promote society’s health. We are not very good when
we wander around in a circle of variables—dependent, independent,
confounded, (and) biased, hoping to discover the magical p< .05 or
confidence limit above 1. We do even worse when we focus on
ill-defined truisms, develop new languages, and presume that we have
discovered a new epidemiology. The future, therefore, of epidemiology
depends on more consequential epidemiological research. (It depends
on) the re-uniting of epidemiology with preventive medicine and public
health, and (on) moving epidemiological studies more rapidly from
descriptive to analytical observational to experimental clinical
trials.
Published July 1996
Postscript 2000
In my
1996 SER address on the sins of epidemiology, I tried to encourage my
fellow epidemiologists to focus more on good, solid, traditional
methods of epidemiology such as including the incubation periods of
the disease, the modes of disease transmission, biological
plausibility, and hypothesis testing. The year 2000 offers new and
important opportunities for epidemiology. The Human Genome Project, I
believe, will soon make it possible to redefine host susceptibility in
the model of host, agent
and environment and thus, will
make epidemiology more important than it has been in the past. The new
molecular methods may provide a technology to better identify agents
as has already been done in the Study of Viral Etiology of Disease
(Kaposi Sarcoma) and good epidemiological studies based on solid
traditional methods may soon bloom in the modern worlds of genetics,
molecular biology and new technology.
What, then, are the
negative problems? First, we still have substantial problems with the
use of epidemiological methods. Many of our colleagues, unfortunately,
are trained in statistical analysis but not in good, solid
methodology. We still are not focused on specific hypothesis testing
but, rather, on collections of data and numerous enthusiastic but
irrelevant analyses.
A good epidemiology
study of “tires” and automobile accidents would have likely identified
the agent without adjusting for the number of people in the car, age
of the driver, type of road pavement, wind speed, outside temperature,
rainfall, snowfall, height and weight of the driver, depressive
symptoms of the driver, relationship between the driver and other
people in the car, whether they were using a cell phone, whether they
were smoking, the association with drug abuse, alcohol consumption,
and their most recent food frequency questionnaire.
The challenge of
epidemiology in the future will be to use our proven methodology to
further evaluate causal hypotheses of disease, to test the efficacies
of intervention, and to provide the scientific basis of good, solid
public health practice. If we do not have good epidemiology, it is
very unlikely that we will have good public health and preventive
medicine. We should remember that the size of the number of
independent variables in a multivariate analysis is most likely
inversely related to the likelihood of finding a true association with
the dependent variable, i.e., disease.
|
