AJPH
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Extract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Purchase Article
Right arrow View Shopping Cart
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Al-Tayyib, A. A.
Right arrow Articles by Turner, C. F.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Al-Tayyib, A. A.
Right arrow Articles by Turner, C. F.
Related Collections
Right arrow Sexual Health
Right arrow Surveys
Right arrow Alcohol
September 2002, Vol 92, No. 9 | American Journal of Public Health 1478-1480
© 2002 American Public Health Association

RESEARCH AND PRACTICE

Effect of Low Medical Literacy on Health Survey Measurements

Alia A. Al-Tayyib, BA, Susan M. Rogers, PhD, James N. Gribble, ScD, Maria Villarroel, MA and Charles F. Turner, PhD

Alia A. Al-Tayyib, Susan M. Rogers, James N. Gribble, Maria Villarroel, and Charles F. Turner are with the Program in Health and Behavior Measurement, Research Triangle Institute, Washington, DC. Charles F. Turner is also with the City University of New York, Queens College and Graduate Center, Flushing, NY.

Correspondence: Requests for reprints should be sent to Alia A. Al-Tayyib, Research Triangle Institute, 1615 M St, NW, Suite 740, Washington, DC 20036 (e-mail: alia{at}rti.org).


   INTRODUCTION
TOP
 INTRODUCTION
METHODS
 RESULTS
 CONCLUSIONS
 References
 
Paper self-administered questionnaires (SAQs) are a standard interviewing technique for surveys of sensitive topics. Although SAQs afford respondents greater privacy than face-to-face interviews, they have several limitations. First, they require literate respondents. Second, they require forms literacy—the ability to select consistent responses, implement general survey instructions, and correctly follow branching or skip instructions. Survey respondents may have difficulty following skip instructions if they are not forms literate.

This brief report examines the relationship between assessed levels of medical literacy, respondent characteristics, and the quality of measurements made in the 1997/98 Baltimore Sexually Transmitted Disease (STD) and Behavior Survey.


   METHODS
TOP
 INTRODUCTION
 METHODS
RESULTS
 CONCLUSIONS
 References
 
Baltimore STD and Behavior Survey
The Baltimore STD and Behavior Survey (BSBS) collected data on sensitive health behaviors, including alcohol and drug use, sexual behaviors, and sexually transmitted diseases, among a probability sample of 1014 adults aged 18 to 45 years residing in Baltimore, Md. For details on sampling design and survey procedures, see Turner et al.1,2 Participants were randomly assigned to 1 of 2 interview modes, audio computer-assisted selfinterview (ACASI) or computer-assisted personal interview (CAPI). Survey questions contained branching or skip patterns—conditional paths through the questionnaire based on answers to specific questions. Participants assigned to the ACASI mode completed the entire questionnaire on a laptop computer. Branching patterns in ACASI were automated, removing the need for the respondent to follow skip instructions. Participants assigned to CAPI were administered the majority of questions by a trained interviewer but completed 2 paper SAQs for the more sensitive behaviors.

Rapid Estimate of Adult Literacy in Medicine
Following completion of the questionnaire, interviewers administered the Rapid Estimate of Adult Literacy in Medicine (REALM) to all participants. The REALM was designed for use in public health settings to identify patients who may need special attention with health care instructions because of low literacy.3 The REALM measures a respondent’s ability to read and correctly pronounce 66 common medical terms. The total number of correctly pronounced words measures each respondent’s REALM score. Scores are collapsed into 4 reading grade range estimates: grade 3 and below, grades 4 through 6, grades 7 through 8, and grade 9 and above.

Paper Self-Administered Questionnaires
The first SAQ contained 10 questions about alcohol and illicit drug use. Respondents who indicated that they had used alcohol were asked 4 CAGE scale questions on alcohol-related problems.4 Respondents indicating that they had never used alcohol were instructed to skip to questions that assessed illicit drug use. The second SAQ contained 12 questions about same-sex attraction and sexual contact, masturbation, forced sexual intercourse, and paid sexual intercourse.5,6 This form also contained skip patterns based on whether or not the respondent had engaged in a particular behavior.

Types of Errors
We identified 3 types of errors made in completing the SAQs. Skip errors included instances in which the respondent did not properly follow the printed skip instructions. Although this type of error may indicate low forms literacy or simple inattention to instructions, it does not suggest that the respondent was unable to comprehend the questions. A second category, logically inconsistent answers, suggests that the respondent was not reading the question and was merely circling an answer at random, or that the respondent did not understand the question. The final category, other errors, included circling more than 1 answer, writing in answers not offered as responses, or writing that the question did not apply.


   RESULTS
TOP
 INTRODUCTION
 METHODS
 RESULTS
CONCLUSIONS
 References
 
Of the 1014 adults who completed the survey, 992 also completed the REALM instrument. Table 1Go presents population-based estimates of medical literacy for the adult population of Baltimore.7 Five percent of adults are estimated to have REALM-assessed literacy scored at the level of grade 3 or below, 6.6% in the grades 4 through 6 range, 16.3% in the grades 7 through 8 range, and 72.1% at the level of grade 9 or higher. Estimates vary significantly by race, self-reported education, income, and sex.


View this table:
[in this window]
[in a new window]
  		TABLE 1— —Medical Literacy as Assessed by the Rapid Estimate of Adult Literacy in Medicine (REALM) in the 1997/98 Baltimore STD and Behavior Survey
 
To assess the effect of medical literacy on our study measurements, we compared the error rates for completion of the SAQ by respondents’ scores on the REALM. Table 2Go summarizes the error rates of the 485 participants who completed the alcohol SAQ and the 487 participants who completed the same-sex sexual intercourse and masturbation SAQ; all of these participants also completed the REALM. Skip errors were the most common type of error across grade range estimates. On the alcohol SAQ, the overall error rate decreased significantly as the literacy level increased (P < .0001). At the literacy range of grade 3 or lower, only 21.4% of respondents were able to complete the alcohol SAQ without errors, compared with 70.5% of those at the level of grade 9 and above. However, skip errors were detected among 25% of those with estimated literacy at grade 9 or above, although only 0.8% answered inconsistently.


View this table:
[in this window]
[in a new window]
  		TABLE 2— —Error Rates for Completion of Self-Administered Questionnaires (SAQs), 1997/98 Baltimore STD and Behavior Survey
 
The layout of the same-sex sexual intercourse and masturbation SAQ was more complex than the alcohol SAQ. The questions assessing respondents’ experience with masturbation contained 2 different skip instructions originating from the same question. For respondents scoring at the level of grade 9 and above, only 53.8% completed the same-sex sexual intercourse and masturbation SAQ without error, whereas 39.7% made skip errors. The proportion of respondents providing logically inconsistent answers on this form was highest among those at the level of grade 3 or below (17.9%).


   CONCLUSIONS
TOP
 INTRODUCTION
 METHODS
 RESULTS
 CONCLUSIONS
References
 
Our findings support a link between low literacy, as assessed by the REALM, and participants’ inability to accurately complete a paper SAQ. We estimate that 28% of Baltimore adults (aged 18 to 45 years) have a REALM-rated literacy at the level of grade 8 or less and that 12% are at the level of grade 6 or less. These results have important implications for the survey measurement of health and other behaviors.

Our data suggest that, although persons with low medical literacy will provide answers on paper self-administered forms, they may respond to questions that they do not completely understand. For example, we found that logically inconsistent answers on the alcohol SAQ were 8 times more likely in the lowest than in the highest medical literacy group (error rate: 7.1% vs 0.8%). A similar pattern was found for the same-sex sexual intercourse and masturbation SAQ. Logically inconsistent answers were found 2.5 times more often with respondents at the level of grade 3 and below (17.9%) than with respondents at the level of grade 9 or higher (6.5%). Measurements made with SAQs are also vulnerable to errors when respondents do not follow questionnaire skip instructions. These error rates increase with lower literacy. Nonetheless, we found that error rates on skip instructions were quite high even among persons whose REALM-assessed literacy tested at the grade 9 and above range (25% to 40%).

These findings provide important evidence for the potential benefits of audio computer-assisted self-interviewing technologies.8–12 ACASI does not require respondent literacy; the respondent listens to the recorded questions and the defined response categories through headphones. ACASI eliminates the requirement that respondents be forms literate by automatically skipping respondents to the next question that is appropriate for them. The trend in survey research to adopt computer-based technologies offers promise for reducing the errors associated with low literacy, thereby improving the quality of survey measurements.


   Acknowledgments
 
Primary support for this research was provided by National Institutes of Health grant R01-HD31067 to C. F. Turner. Additional support was provided by the Research Triangle Institute and by grant R01-MH56318.

The authors would like to thank the members of the 1997/98 Baltimore STD and Behavior Survey team for their contributions to the study and the survey operations staff of the Research Triangle Institute for their fielding of the survey. We would also like to thank Dr. Terry Davis and her colleagues for the use of their literacy assessment instrument.

Human Participant Protection
The protocol for this study was approved by institutional review boards at the Research Triangle Institute (RTI) and the Johns Hopkins Medical Institutions (JHMIs).


   Footnotes
 
Peer Reviewed

A. A. Al-Tayyib analyzed the data and wrote the brief. S. M. Rogers supervised data analysis and contributed to the writing. J. N. Gribble supervised the initial conception of the data analysis plan. M. Villarroel duplicated the data analysis for quality control purposes. C. F. Turner conceived and designed the 1997/98 Baltimore Sexually Transmitted Diseases and Behavior Survey and contributed to the writing of the brief. All authors approved the final version of this brief.

Accepted for publication November 12, 2001.


   References
TOP
 INTRODUCTION
 METHODS
 RESULTS
 CONCLUSIONS
 References
 
1. Turner CF, Rogers SM, Miller HG, et al. Untreated gonococcal and chlamydial infection in a probability sample of adults. JAMA. 2002; 287:726–733.[Abstract/Free Full Text]

2. Turner, CF, Gribble JN, Al-Tayyib AA, Chromy JR. Falsification in epidemiologic surveys: detection and remediation. Paper presented at ORI-NIH-AAAS-AAMC-NSF Conference on Research Integrity, Bethesda, Md, November 18–20, 2000.

3. Davis TC, Long SW, Jackson RH, et al. Rapid estimate of adult literacy in medicine: a shortened screening instrument. Fam Med. 1993;25:391–395.[Medline]

4. Mayfield D, McLeod G, Hall P. The CAGE questionnaire: validation of a new alcoholism screening instrument. Am J Psychiatry. 1974;131:1121–1123.[Abstract/Free Full Text]

5. Laumann EO, Gagnon JH, Michael RT, Michaels S. The Social Organization of Sexuality. Chicago, Ill: University of Chicago Press; 1994.

6. Johnson AM, Wadsworth J, Wellings K, Field J. Sexual Attitudes and Lifestyles. Oxford, England: Blackwell Scientific Publications; 1994.

7. US Bureau of the Census, Population Division, Population Estimates Program. Population Estimates for Cities With Populations of 100,000 and Greater; July 1, 1998. Washington, DC: US Bureau of the Census; 1999. Publication series SU-98–1.

8. Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science. 1998;280:867–873.[Abstract/Free Full Text]

9. O’Reilly JM, Hubbard ML, Lessler JT, Biemer PP, Turner CF. Audio and video computer-assisted selfinterviewing: preliminary tests of new technologies for data collection. Journal of Official Statistics. 1994;10:197–214.

10. Turner CF, Forsyth BH, O’Reilly JM, et al. Automated self-interviewing and the survey measurement of sensitive behaviors. In: Couper MP, Baker RP, Bethlehem J, et al, eds. Computer-Assisted Survey Information Collection. New York, NY: John Wiley & Sons, Inc; 1998: 455–473.

11. Rogers SM, Gribble JN, Turner CF, Miller HG. Entretiens Autoadministré sur ordinateur et mesure des comportements sensibles [Computerized selfinterviewing and the measurement of sensitive behaviors]. Population. 1999;54(2):231–250.

12. Gribble JN, Miller HG, Rogers SM, Turner CF. Interview mode and measurement of sexual behaviors: methodological issues. J Sex Res. 1999;36:16–24.





This Article
Right arrow Extract Freely available
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Purchase Article
Right arrow View Shopping Cart
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Al-Tayyib, A. A.
Right arrow Articles by Turner, C. F.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Al-Tayyib, A. A.
Right arrow Articles by Turner, C. F.
Related Collections
Right arrow Sexual Health
Right arrow Surveys
Right arrow Alcohol

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
Copyright © 2002 by the American Public Health Association