|
|
||||||||
RESEARCH AND PRACTICE |
Ross C. Brownson, Jen Jen Chang, and Amy A. Eyler are with the Department of Community Health and Prevention Research Center, Saint Louis University School of Public Health, St Louis, Mo. Barbara E. Ainsworth and Karen A. Kirtland are with the Prevention Research Center, University of South Carolina School of Public Health, Columbia. Brian E. Saelens is with the Department of Pediatrics, Division of Psychology, Childrens Hospital Medical Center, Cincinnati, Ohio. James F. Sallis is with the Department of Psychology, San Diego State University, San Diego, Calif.
Correspondence: Requests for reprints should be sent to Ross C. Brownson, Dept of Community Health and Prevention Research Center, Saint Louis University School of Public Health, Salus Center Room 469, 3545 Lafayette Ave, St Louis, MO 63104 (e-mail: brownson{at}slu.edu).
| ABSTRACT |
|---|
|
|
|---|
Objectives. We tested the reliability of 3 instruments that assessed social and physical environments.
Methods. We conducted a testretest study among US adults (n = 289). We used telephone survey methods to measure suitableness of the perceived (vs objective) environment for recreational physical activity and nonmotorized transportation.
Results. Most questions in our surveys that attempted to measure specific characteristics of the built environment showed moderate to high reliability. Questions about the social environment showed lower reliability than those that assessed the physical environment. Certain blocks of questions appeared to be selectively more reliable for urban or rural respondents.
Conclusions. Despite differences in content and in response formats, all 3 surveys showed evidence of reliability, and most items are now ready for use in research and in public health surveillance.
| INTRODUCTION |
|---|
|
|
|---|
The physical, or built, environment is important in providing cues and opportunities for activity,8 and it is associated with rates of physical activity in intervention studies and in large population-based surveys.9 Support for the importance of the environment is derived from 2 distinct literatures. A review of 19 studies in the physical activity and health literature showed consistent associations of accessibility of recreational facilities, opportunities to be active, and certain aesthetic qualities with physical activity in adults.10 Researchers in the transportation and urban planning fields have examined the relationship between community design variables and walking or cycling for transportation. Fourteen studies have consistently shown that people walk and cycle more when their neighborhoods have higher residential density, a mixture of land uses (e.g., shops are within walking distance of homes), and connected streets (e.g., gridlike pattern instead of many cul-de-sacs).11 Other community design characteristics, such as the condition of sidewalks, the presence of bike paths, street design, traffic volume and speed, and crime, are hypothesized to be related to physical activity12,13 but have not been systematically examined. In addition, rural areas have important differences from urban areas in their activity-related design features1416 and are generally understudied.11
Multiple questionnaires have been developed to assess physical activitymeasurement properties (i.e., reliability/validity) are documented for many of these. For example, Ainsworth et al.17 reported on the measurement properties of 39 questionnaires, and Kriska and Caspersen18 described the validity, reliability, and comprehensiveness of 32 instruments. In contrast, considering the apparent importance of the built environment, there is limited information in the literature on how best to measure various aspects, such as the presence of well-maintained sidewalks or whether shopping venues are within walking distance.19 One method of measuring the perceived physical environment is through population-based surveys and surveillance systems.20 Individual responses from these surveys can be aggregated to identify patterns in important design/neighborhood features (e.g., lack of access to sidewalks in rural areas) and to determine associations between these design features and behavior.2123 As yet, it is unclear whether the objective environment (e.g., actual counts of traffic) or the perceived environment (e.g., an individuals self-reported perception of crime in his/her neighborhood) is more important in explaining physical activity.10,11
As measures of perceived environments are developed, it is important to ensure that they can be administered by multiple modes (e.g., self and interviewer administered) and are reliable for broad populations. Our study reports the results of reliability testing of 3 instruments among urban and rural residents across the United States. A major focus of the instruments tested was the assessment of environmental characteristics that are believed to be related to recreational physical activity and nonmotorized transportation, although some instruments assessed other related variables.
| METHODS |
|---|
|
|
|---|
Because the purpose of our study was to determine testretest reliability, respondents who completed the survey were asked if they would be willing to complete the survey again in 7 to 21 days, and they were asked for the most convenient time to call for the resurvey. The second calls were made within the 7-to-21-day time frame, and the survey was readministered. This time frame is often used in testretest studies because it is a long enough period so that respondents are unlikely to remember their answers to the original survey, yet the time frame is short enough so that changes in behavior (e.g., seasonal changes in physical activity) are unlikely to have occurred. Each survey participant was assigned randomly to 1 of the 3 questionnaires.
Questionnaires
Three questionnaires were used: the San Diego instrument, the South Carolina instrument, and the St Louis instrument. Each of these had been previously tested for some psychometric properties but had not been tested for reliability side by side across a nationwide population. All 3 questionnaires, decribed in the following paragraphs, used the same sociodemographic questions (i.e., race/ethnicity, age, gender, education level, income, employment status). A copy of each questionnaire is available from the lead investigator at each site: San DiegoJ.F.S. (http://www.rohan.sdsu.edu/faculty/sallis/index.html), South CarolinaB.E.A. (http://prevention.sph.sc.edu), and St LouisR.C.B. (http://prc.slu.edu).
The San Diego instrument (also called the Neighborhood Environment Walkability Survey). This 98-question instrument was developed by Sallis et al. to determine the perception of neighborhood design features hypothesized to be related to physical activity. The questionnaire includes questions about types of residences (to assess density), proximity of stores and facilities in the neighborhood, perceived access to these places, street characteristics (to assess connectivity), facilities for walking and cycling, neighborhood aesthetics, and safety regarding traffic and crime. The San Diego instrument was originally developed for self-administration and was therefore adapted for telephone administration in our study. A reliability study of a self-administered version of this instrument was completed in San Diego.27
The South Carolina instrument. This 61-question instrument was developed by Ainsworth et al. and includes an assessment of the physical and social environments, including perceptions of the community environment (e.g., whether the neighborhood is pleasant), safety, access to recreation and shopping destinations, and conditions of the neighborhood and facilities. Thirteen items focus on the neighborhood, which is defined as a half-mile radius or a 10-minute walk from the respondents home, and 13 items focus on the community, which is defined as a 10-mile radius or a 20-minute drive from the respondents home. Additional physical activity questions from the BRFSS incorporate an assessment of employment activity as well as moderate and vigorous physical activities and global walking behaviors. This instrument was previously tested for reliability and validity among 1200 adults who lived in Sumter County, SC.28
The St. Louis instrument. This 104-question survey was developed by Brownson et al. to measure physical activity and environmental influences on physical activity across the United States.22,29,30 Several constructs in the St Louis instrument were used to develop and evaluate physical activity interventions in rural settings.14,31 The questionnaire includes a detailed assessment of walking behavior, places to walk, barriers to being physically active, neighborhood infrastructure for walking and cycling, perceptions about places for walking, social assets, social support for physical activity, community assets, policy attitudes, and sedentary behaviors. An earlier version of this instrument was tested for reliability in a US sample of ethnically diverse women aged 40 years and older.32
Data Collection
Telephone calls were made in April and May 2002 by experienced interviewers who had at least 8 hours of specific training for this project. Calls were made between 5:00 P.M. and 9:00 P.M. on weekdays and between 12:00 P.M. and 6:00 P.M. on weekends. The person who answered the phone, if eligible, responded to the survey. If the person who answered the phone was under 18, he or she was asked to give the phone to an available adult who lived in the household. The baseline response rate for the original survey was 36.3%, which was calculated with the method of the Council of American Survey Research Organizations,33 and the retest survey had a response rate of 63.9%. The average interview administration times for each of 3 instruments were as follows: San Diego24 minutes, South Carolina25 minutes, and St Louis30 minutes. These administration times provide only a rough estimate of time needed to complete surveys of environmental features, because questionnaires varied in their number of questions related to factors other than the physical environment (e.g., the St Louis instrument included a substantial block of questions about social and community assets).
Analyses
After cleaning and editing the data, the reliability of each variable from time 1 to time 2 was assessed with the 1-way model intraclass correlation coefficient (ICC).34 The ICC is derived from a 1-way analysis of variance, and it represents the proportion of total variation accounted for by the variability between, rather than within, subjects. As a rough guide, we followed the adjectival ratings suggested by Landis and Koch35 in the following categories: 1.0 to 0.8 (almost perfect agreement), 0.8 to 0.6 (substantial agreement), 0.6 to 0.4 (moderate agreement), 0.4 to 0.2 (fair agreement), and 0.2 to 0.0 (poor agreement). Agreement for each of the 3 questionnaires also was considered separately for urban versus rural respondents, because earlier studies have shown that patterns in these environmental characteristics can vary widely depending on urban or rural residence.15,16
| RESULTS |
|---|
|
|
|---|
|
|
|
|
The highest proportion of questions within the St Louis questionnaire (Table 4
) showed moderate reliability (n = 12). Only 2 questions ("most-liked feature of the walking facilities" and "hours spent driving for delivery/picking up kids/errands per week") resulted in poor agreement. Except for the question about hours spent using a computer, each of the questions that attempted to measure sedentary behaviors showed only fair or poor agreement. Differences in reliabilities for urban versus rural respondents were more distinct for the St Louis instrument than for the other 2 questionnaires. For questions that assessed the community environment, 1 question ("least-liked feature of the walking facility") was more reliable for urban respondents, and all other questions showed higher reproducibility among rural respondents. Of the 5 questions that assessed sedentary behaviors, 4 showed higher reliability for urban compared with rural respondents, and 1 question in this domain ("hours spend driving for delivery/picking up kids/errands") showed no urbanrural variation. Finally, for neighborhood safety, 7 of 8 questions were more reliable for rural compared with urban respondents.
| DISCUSSION |
|---|
|
|
|---|
The other main source of environmental measures is from survey or surveillance data on individuals that can be aggregated to some larger unit (e.g., zip code) and compared across subgroups. The 3 instruments used in our study are useful for this type of data collection and analysis. Although each instrument was designed for a slightly different purpose, most of the variables were reasonably reliable in a diverse sample of adults. In spite of differences in content and response formats, all 3 surveys showed evidence of reliability, and most items are now ready for use in research and in public health surveillance. Several patterns in our data deserve mention:
When determining which questionnaire scales to use in a particular study, there is a trade-off between the ability to comprehensively measure all domains and the feasibility of collecting data efficiently. It is necessary to match environmental variables with the physical activity outcomes of interest, and very specific hypotheses may need to be developed. For example, walking for transportation is likely to be related to the presence of shops nearby, and walking for recreation may be more related to neighborhood aesthetics. Bicycling is expected to be related to accessibility of cycling facilities, and other types of recreational physical activity may be related to presence, condition, and accessibility of recreational facilities. The 3 questionnaires we evaluated can assess a wide range of environmental variables that allow researchers to test multiple hypotheses.
The next research priority is to test hypotheses about the relationship between environmental variables and physical activity. Because it is not clear whether perceived or objectively measured environmental variables provide more explanatory power, the use of triangulationapplying multiple methods of data collection to determine points of concordance or disagreement44,45is recommended. A broad range of populations should be studied for several reasons; for example, children and older adults are likely to do physical activity in different settings. Both cross-sectional and longitudinal studies are needed in multiple settings that range from urban to rural locales. To adequately explain physical activity, researchers should examine the separate and interactive contributions of psychological, social, and environmental variables.12 As consensus is reached on the most important correlates or predictors of physical activity, these variables can be incorporated into national surveillance systems.
There are several limitations to our study that deserve mention. We relied on selfreported telephone survey data for which there are several potential biases (e.g., possible underrepresentation of lower socioeconomic status segments of the population).4648 Our questions about the environment were selfreported and did not include separate objective measures that would allow assessment of validity (i.e., presence of some "gold standard"). However, 1 recent study found statistically significant associations between selfreported and objectively measured (with geographic information systems) characteristics of trails that may influence physical activity.43 We do not intend to imply that perceived environment measures are preferred over objective measures. At this early phase in this field of research, it is important to evaluate both perceived and objective measures of the environment as they relate to physical activity. Our response rate for the initial survey was lower than anticipated. However, because our study was not developed to measure prevalence, and because the follow-up response rate was reasonable (64%), our reliability results should not be subject to substantial bias. Other similar reliability studies of questionnaires on physical activity have shown response rates from 13% to 54%,4951 and many reliability studies have relied on convenience samples. Both the length of the survey/completion time and the content areas may be factors in the low baseline response rate.
Surveillance of chronic diseases has focused primarily on the diseases themselves until recently, when national systems began tracking behavioral risk factors and changes in preventive health practices.25,5254 Our study suggests that numerous dimensions of the physical environment can be measured reliably with telephone survey methods. Multisite collaborations such as ours allow for the testing of multiple instruments simultaneously. Surveillance systems need to begin capturing key aspects of the physical and social environments in addition to the main focus on the behavior of physical activity.20,54
| CONCLUSIONS |
|---|
|
|
|---|
| Acknowledgments |
|---|
Human Participant Protection
Human subjects approval was obtained from the Saint Louis University institutional review board.
| Footnotes |
|---|
Accepted for publication March 12, 2003.
| References |
|---|
|
|
|---|
2. McGinnis JM. The public health burden of a sedentary lifestyle. Med Sci Sports Exerc. 1992;6(suppl):S196S200.
3. McGinnis JM, Foege WH. Actual causes of death in the United States. JAMA. 1993;270:22072212.[Abstract]
4. Powell KE, Blair SN. The public health burden of sedentary living habits: theoretical but realistic estimates. Med Sci Sports Exerc. 1994;26:851856.[ISI][Medline]
5. US Dept of Health and Human Services. Healthy People 2010. Volume II. Conference Edition. Washington, DC: US Dept of Health and Human Services; 2000.
6. Centers for Disease Control and Prevention. Physical activity trendsUnited States, 19901998. MMWR Morb Mortal Wkly Rep. March 9, 2001;50(09):166169.
7. World Health Organization. Promoting Physical Activity for Health and Wellbeing: An Achievable Collective Challenge. Available at: http://www.who.int/hpr/physactiv/publications.htm. Accessed June 29, 2002.
8. Giles-Corti B, Donovan RJ. The relative influence of individual, social and physical environment determinants of physical activity. Soc Sci Med. 2002;54(12):17931812.
9. Sallis JF, Bauman A, Pratt M. Environmental and policy interventions to promote physical activity. Am J Prev Med. 1998;15(4):379397.[ISI][Medline]
10. Humpel N, Owen N, Leslie E. Environmental factors associated with adults participation in physical activity. A review. Am J Prev Med. 2002;22(3):188199.[ISI][Medline]
11. Saelens BE, Sallis JF, Frank LD. Environmental correlates of walking and cycling: findings from the transportation, urban design and planning literatures. Ann Behav Med. 2002;25:8091.
12. Booth SL, Sallis JF, Ritenbaugh C, et al. Environmental and societal factors affect food choice and physical activity: rationale, influences, and leverage points [discussion]. Nutr Rev. 2001;59(3 Pt 2):S21S39; S57S65.[ISI][Medline]
13. Frank L, Engelke P, Schmid T. Health and Community Design. The Impact of the Built Environment on Physical Activity. Washington, DC: Island Press; 2003.
14. Brownson RC, Housemann RA, Brown DR, et al. Promoting physical activity in rural communities: walking trail access, use, and effects. Am J Prev Med. 2000;18:235241.[ISI][Medline]
15. Parks SE, Housemann RA, Brownson RC. Differential correlates of physical activity in urban and rural adults of various socioeconomic backgrounds in the United States. J Epidemiol Community Health. 2003;57(1):2935.
16. Wilcox S, Castro C, King AC, Housemann R, Brownson RC. Determinants of leisure time physical activity in rural compared with urban older and ethnically diverse women in the United States. J Epidemiol Community Health. 2000;54:667672.
17. Ainsworth BE, Montoye HJ, Leon AS. Methods of assessing physical activity during leisure and work. In: Bouchard C, Shephard RJ, Stephens T, eds. Physical Activity, Fitness, and Health: International Proceedings and Consensus Statement. Champaign, Ill: Human Kinetics; 1994:146159.
18. Kriska AM, Caspersen CJ. A collection of physical activity questionnaires for health-related research. Med Sci Sports Exerc. 1997;29(suppl):S3S205.
19. Bauman A, Sallis JF, Owen N. Environmental and policy measurement in physical activity research. In: Welk GJ, ed. Physical Activity Assessments for Health-Related Research. Champaign, Ill: Human Kinetics; 2002.
20. Baker EA, Brennan LK, Brownson RC, Housemann RA. Measuring the determinants of physical activity in the community: current and future directions. Res Q Exerc Sport. 2000;71:146158.
21. Bauman A, Smith B, Stoker L, Bellew B, Booth M. Geographical influences upon physical activity participation: evidence of a "coastal effect." Aust N Z J Public Health.1999;23:322344.[ISI][Medline]
22. Brownson RC, Baker EA, Housemann RA, Brennan LK, Bacak SJ. Environmental and policy determinants of physical activity in the United States. Am J Public Health. 2001;91(12):19952003.
23. Sallis JF, Hovell MF, Hofstetter CR, et al. Distance between homes and exercise facilities related to frequency of exercise among San Diego residents. Public Health Rep. 1990;105(2):179185.[ISI][Medline]
24. Gentry EM, Kalsbeek WD, Hogelin GC, et al. The behavioral risk factor surveys: II. Design, methods, and estimates from combined state data. Am J Prev Med. 1985;1(6):914.[Medline]
25. Remington PL, Smith MY, Williamson DF, Anda RF, Gentry EM, Hogelin GC. Design, characteristics, and usefulness of state-based behavioral risk factor surveillance: 19811987. Public Health Rep. 1988;103:366375.[ISI][Medline]
26. US Dept of Commerce. US Census Bureau. Available at: http://www.census.gov. Accessed July 14, 2002.
27. Saelens BE, Sallis JF, Black JB, Chen D. Neighborhood-based differences in physical activity: an environment scale evaluation. Am J Public Health. 2003;93:15521558.
28. Ainsworth BA, Bassett DRJ, Strath SJ, et al. Comparison of three methods of measuring time spent in physical activity. Med Sci Sports Exerc. 2000;32(suppl 9):S457S464.[ISI][Medline]
29. Eyler AA, Brownson RC, Donatelle RJ, King AC, Brown D, Sallis JF. Physical activity social support and middle- and older-aged minority women: results from a US survey. Soc Sci Med. 1999;49:781789.
30. King AC, Castro C, Wilcox S, Eyler AA, Sallis JF, Brownson RC. Personal and environmental factors associated with physical inactivity among different racial-ethnic groups of US middle-aged and older-aged women. Health Psychol. 2000;19(4):354364.[ISI][Medline]
31. Brownson RC, Schmid TL, King AC, et al. Support for policy interventions to increase physical activity in rural Missouri. Am J Health Promotion. 1998;12(4):263266.[ISI][Medline]
32. Brownson RC, Eyler AA, King AC, Shyu Y-L, Brown DR, Homan SM. Reliability of information on physical activity and other chronic disease risk factors among US women aged 40 years or older. Am J Epidemiol. 1999;149:379391.
33. CASRO Task Force on Completion Rates. On the Definitions of Response Rates. Special Report. New York, NY: Council of American Survey Research Organizations; 1982.
34. Fisher RA.Statistical Methods for Research Workers. 14th ed. New York, NY: Hafner; 1973.
35. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159174.[ISI][Medline]
36. King AC, Jeffery RW, Fridinger F, et al. Environmental and policy approaches to cardiovascular disease prevention through physical activity: issues and opportunities. Health Educ Q. 1995;22(4):499511.[ISI][Medline]
37. Webb EJ, Campbell DT, Schwartz RD, Sechrest L. Unobtrusive Measures: Non-Reactive Research in the Social Sciences. Chicago, Ill: Rand McNally & Co.; 1966.
38. Sechrest L, Phillips M. Unobtrusive measures: an overview. In: Sechrest L, ed. Unobtrusive Measurement Today. San Francisco, Calif: Jossey-Bass; 1979:117.
39. Cheadle A, Wagner E, Koepsell T, et al. Environmental indicators: a tool for evaluating community-based health-promotion programs. Am J Prev Med. 1992;8(6):345350.[ISI][Medline]
40. Cheadle A, Sterling TD, Schmid TL, Fawcett SB. Promising community-level indicators for evaluating cardiovascular health-promotion programs. Health Educ Res. 2000;15(1):109116.
41. Pikora T, Bull F, Jamrozik K, Knuiman M, Giles-Corti B, Donovan R. Developing a reliable audit instrument to measure the physical environment for physical activity. Am J Prev Med. 2002;23(3):187.[ISI][Medline]
42. Moore DA, Carpenter TE. Spatial analytical methods and geographic information systems: use in health research and epidemiology. Epidemiol Rev. 1999;21(2):143161.
43. Troped PJ, Saunders RP, Pate RR, Reininger B, Ureda JR, Thompson SJ. Associations between self-reported and objective physical environmental factors and use of a community rail-trail. Prev Med. 2001;32(2):191200.[ISI][Medline]
44. Denzin NK.The Research Act in Sociology. London, England: Butterworth; 1970.
45. Steckler A, McLeroy KR, Goodman RM, Bird ST, McCormick L. Toward integrating qualitative and quantitative methods: an introduction. Health Educ Q. 1992;19(1):18.[ISI][Medline]
46. Marcus AC, Crane LA. Telephone surveys in public health research. Med Care. 1986;24(2):97112.[ISI][Medline]
47. Centers for Disease Control and Prevention. Using Chronic Disease Data: A Handbook for Public Health Practitioners. Atlanta, Ga: Centers for Disease Control and Prevention; 1992.
48. Ford ES. Characteristics of survey participants with and without a telephone: findings from the third National Health and Nutrition Examination Survey. J Clin Epidemiol. 1998;51(1):5560.[ISI][Medline]
49. Pols MA, Peeters PH, Ocke MC, et al. Relative validity and repeatability of a new questionnaire on physical activity. Prev Med. 1997;26(1):3743.[ISI][Medline]
50. Blair SN, Dowda M, Pate RR, et al. Reliability of long-term recall of participation in physical activity by middle-aged men and women. Am J Epidemiol. 1991;133(3):266275.
51. Chasan-Taber L, Erickson JB, McBride JW, Nasca PC, Chasan-Taber S, Freedson PS. Reproducibility of a self-administered lifetime physical activity questionnaire among female college alumnae. Am J Epidemiol. 2002;155(3):282289.
52. Berkelman RL, Buehler JW. Public health surveillance of non-infectious chronic diseases: the potential to detect rapid changes in disease burden. Int J Epidemiol. 1990;19(3):628635.
53. Thacker SB, Stroup DF, Rothenberg RB. Public health surveillance for chronic conditions: a scientific basis for decisions. Stat Med. 1995;14(57):629641.[ISI][Medline]
54. Macera CA, Pratt M. Public health surveillance of physical activity. Res Q Exerc Sport. 2000;71(suppl 2):S97S103.[ISI][Medline]
This article has been cited by other articles:
![]() |
K. S. Morris, E. McAuley, and R. W. Motl Self-efficacy and environmental correlates of physical activity among older women and women with multiple sclerosis Health Educ. Res., August 1, 2008; 23(4): 744 - 752. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. D. Deshpande, E. A. Baker, S. L. Lovegreen, and R. C. Brownson Environmental Correlates of Physical Activity Among Individuals With Diabetes in the Rural Midwest Diabetes Care, May 1, 2005; 28(5): 1012 - 1018. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |