Principles of Epidemiology for Public Health (EPID600)
Sources of error: Information bias
Victor J. Schoenbach, PhD home page
Department of Epidemiology
Gillings School of Global Public Health
University of North Carolina at Chapel Hill
www.unc.edu/epid600/
3/22/2011
Sources of error: Information bias
1
Abort, Retry, Fail
“Tips for safer drives: Never turn off a
PC or accessories while the computer is
on or the disk is active.”
— USA Today
[PC Magazine, 10/3/1996]
11/5/2001
Sources of error: Information bias
2
Chapter 1
THE HISTORIAN'S TASK:
Insight into the future
History, a record of things left behind by
past generations, started in 1815. Thus
we should try to view historical times as
the behind of the present.
Anders Henriiksson (ed), Non Campus Mentis, NY,
Workman Publishing Co., 2003
Non Campus Mentis
“History, as we know, is always bias,
because human beings have to be
studied by other human beings, not by
independent observers of another
species.”
Anders Henriiksson (ed), Non Campus Mentis, NY,
Workman Publishing Co., 2003, chapter 1
Information bias
Information bias: a systematic distortion
or error that arises from the procedures
used for classification or measurement
of the disease, the exposure, or other
relevant variables.
3/22/2011
Sources of error: Information bias
5
Information bias
• Classification or measurement
• Differential or nondifferential bias
• Direction of bias
• Misclassification of covariables
3/22/2011
Sources of error: Information bias
6
Classification or measurement
• Data for epidemiologic studies consist
of classifications (e.g., “hypertensive” vs.
“normotensive”) or measurements (e.g., 120
mmHg systolic BP).
• Possible sources of measurement or
classification error include instrumentation,
laboratories, records, respondents; data
collectors, managers, analysts, and
interpreters.
3/22/2011
Sources of error: Information bias
7
Sources of measurement error
Respondent (interview, questionnaire):
• inability to understand, recall, articulate;
• unwillingness to disclose
• social desirability influences
Can be influenced by wording of questions
and how they are asked.
3/22/2011
Sources of error: Information bias
8
Example of misunderstanding
• Medico – Não consigo encontrar o
motivo das suas dores, meu caro. Só
pode ser por causa da bebida.
• Paciente – Não tem importãncia,
doutor. Eu volto outro dia que o senhor
estiver sóbrio.
De Luciana V. Paiva, Osasco - SP, en Bom Humor Nosso E Dos Leitores”,
Almanaque Brasil de Cultura Popular. Maio 2001;3(26)
(almanaquebrasil@uol.com.br). Exemplar de quem viaja TAM.
9
How not to ask questions
“Has anyone ever tried to give you the
mistaken idea that sex intercourse is
necessary for the health of the young man?
(from a survey by the NC state health officer, circa 1926,
summarized in Kinsey et al., 1948)
Can you guess the right answer?
3/22/2011
Sources of error: Information bias
10
Respondent cognitive processes
• Respondent cognitive processes: interpretation,
recall, judgment formation, response formatting,
editing
• Qualitative research on response processes, e.g.:
• “What types of physical activity or exercise did you
perform during the past month?”
• “What did you think we meant when we said ‘physical
activity’?”
• “Which, if any, of the following would you (also)
consider to be physical activity?
www.minority.unc.edu/institute/2000/materials/slides/RichardWarnecke-2000-06-08.ppt
3/27/2007
Sources of error: Information bias
11
Cognitive testing - 2
Recalling and retrieving – Retrieval probes:
• Recall strategy
• Recall interval
• Search strategy (proximal, distal, anywhere)
• Long term recall - link to events to help
remember
• Recall frame of reference--what kinds of
things helped you remember?
www.minority.unc.edu/institute/2000/materials/slides/RichardWarnecke-2000-06-08.ppt
3/27/2007
Sources of error: Information bias
12
Surveys and Questionnaires
• Survey validation
• Pretesting (wording, item sequence, time)
• Pilot testing (all steps - procedure, item
performance)
• Translation validation
3/22/2011
Sources of error: Information bias
13
Sources of measurement error
• Data collector: unclear or ambiguous
questions, lack of a neutral demeanor,
insufficiently conscientious, inaccurate
transcription, fraud
3/27/2007
Sources of error: Information bias
14
Sources of measurement error
• Data managers: inaccurate transcription,
mis-reading, miscoding, programming
errors
• Data analysts: variable coding and
programming errors
• Data interpreters: inadequate appreciation
of the characteristics of the measure or of
the relations being studied
11/5/2001
Sources of error: Information bias
15
Techniques for avoiding data collection errors
• Precise operational definitions of variables
• Detailed measurement protocols
• Repeated measurements on key variables
• Training, certification, and re-certification
• Data audits (of interviewers, of data centers)
• Data cleaning – visual, computer
• Re-running all analyses prior to publication
11/5/2001
Sources of error: Information bias
16
Validation and agreement
• Sensitivity and specificity – used to evaluate
classifications
• When no validation standard, we measure
agreement
• Measures of agreement often correct for
“chance”
11/5/2009
Sources of error: Information bias
17
Information bias – differential
or non-differential
• Important question for any kind of bias –
are error processes different for groups
being compared
• If no, “non-differential”
• If yes, “differential”
• Has implications for direction of bias
• In general, non-differential is safer
11/5/2001
Sources of error: Information bias
18
Direction of bias
• “Upward”
• “Downward”
• “Towards the null”
• “Away from the null”
Null = 0 (for differences)
1 (for ratios)
3/22/2011
Sources of error: Information bias
19
Direction of bias
In simple situation, information bias is
towards the null IF:
1. Dichotomous exposure and disease
2. Non-differential misclassification with
both sensitivity and specificity each
greater than 0.5; AND
3. Errors in one variable are independent
of errors in the other
11/5/2001
Sources of error: Information bias
20
Errors in covariables
• It is almost always important to control for
other variables (e.g., age)
• Errors in measurement of these variables
hamper attempts to control for them
• Direction of bias is generally unpredictable
11/5/2001
Sources of error: Information bias
21
Classic studies – 6 degrees of separation?
• Classic experiment by Yale psychologist Stanley
Milgram asked people in Kansas to forward a
letter to a target person in Massachusetts
• If did not know target person, then send it to
someone they thought might know him
• Milgram’s 1967 paper reported that it only took 5
jumps, on average, for letters to arrive
3/22/2011
Sources of error: Information bias
22
Selection bias in a classic study
According to Judith Kleinfeld,
psychologist at the University of Alaska,
Fairbanks, archives reveal that only 30%
of the letters actually reached their
destination!
(Gewolb, Josh. Random samples. Science 26
October 2001;294:777. See Kleinfeld, Judith S.
Society. Jan/Feb2002; 39(2):61-66)
3/29/2005
Sources of error: Information bias
23
Dr. Kinsey and the Institute for Sex Research
Alfred S. Kinsey (photograph from Wardell B. Pomeroy,
Dr. Kinsey and the Institute for Sex Research)
24
Sex research in the mid-20th century
Alfred S. Kinsey (photograph from Wardell B. Pomeroy,
Dr. Kinsey and the Institute for Sex Research)
25
Kinsey et al. on selection and information bias
(Alfred C. Kinsey, Wardell B. Pomeroy, Clyde E.
Martin. Sexual behavior in the human male, Phila,
W.B. Saunders, 1948)
11/7/2005
Alfred S. Kinsey (photograph from Wardell B. Pomeroy,
Dr. Kinsey and the Institute for Sex Research)
26
4/2/2002
Sources of error: Information bias
27
Real-life example: Quit for Life
• Randomized trial of smoking cessation
interventions
• Self-reported “In the past 7 days, have you
smoked a cigarette, even a puff?”
• Attempted (unsuccessfully) to validate with
saliva cotinine
• People who did not give a time to be called
for validation had very high quit rates!
3/22/2011
Sources of error: Information bias
28
Kinsey biography by Wardell Pomeroy
Pomeroy, Wardell B. Dr. Kinsey and the Institute for Sex Research
NY, Signet / New American Library, 1972: p136
29
Non Campus Mentis
“Hindsight, after all, is caused by a
lack of foresight.”
Anders Henriiksson (ed), Non Campus Mentis, NY,
Workman Publishing Co., 2003, chapter 1