HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) All Versions of this Article: AJPH.2005.063693v1 96/3/473    most recent Submit a response Purchase Article View Shopping Cart Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Joffe, M. Articles by Mindell, J. Search for Related Content PubMed PubMed Citation Articles by Joffe, M. Articles by Mindell, J. Related Collections Epidemiology Health Policy Prevention Socioeconomic Factors

Michael Joffe, PhD, MD, MSc(Econ) and Jennifer Mindell, MB, BS, PhD, FFPH

Michael Joffe is with Imperial College, London, England. At the time of writing, Jennifer Mindell was also with Imperial College, London.

Correspondence: Requests for reprints should be sent to Michael Joffe, PhD, MD, MSc(Econ), Dept of Epidemiology and Public Health, Imperial College, St Mary’s Campus, Norfolk Pl, London W2 1PF United Kingdom (e-mail: m.joffe{at}imperial.ac.uk).

	ABSTRACT

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 

Causal diagrams are rigorous tools for controlling confounding. They also can be used to describe complex causal systems, which is done routinely in communicable disease epidemiology. The use of change diagrams has advantages over static diagrams, because change diagrams are more tractable, relate better to interventions, and have clearer interpretations.

Causal diagrams are a useful basis for modeling. They make assumptions explicit, provide a framework for analysis, generate testable predictions, explore the effects of interventions, and identify data gaps. Causal diagrams can be used to integrate different types of information and to facilitate communication both among public health experts and between public health experts and experts in other fields. Causal diagrams allow the use of instrumental variables, which can help control confounding and reverse causation.

	INTRODUCTION

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 
CAUSAL DIAGRAMS ARE a useful way of summarizing information not only for presentation and communication but also for analysis. They can specify causal relationships for modeling in a way that is different from traditional epidemiology, in which "modeling" tends to be used in the sense of statistical modeling (an inductive approach). There is potential for more use both of diagrammatic ways of organizing causal relationships in complex systems and of a priori modeling that specifies causal pathways, an approach that is well established in other disciplines, such as air pollution modeling and management studies.^1,2

	HOW DIAGRAMMATIC METHODS HAVE BEEN USED

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 
Causal diagrams that indicate the relationship between variables have been developed in recent years to help interpret epidemiological relationships.^3,4 Because the diagrams depict links that are causal and not merely associational,^5–7 they lend themselves to the analysis of confounding and selection effects. The theory of directed acyclic graphs has developed formal rules for identifying variables that must be measured and controlled to obtain unconfounded effect estimates. They have been shown to be equivalent to algebraic formulations.³ Thus, they are not only visually appealing but also logically rigorous, and they can help with planning data collection and analysis, communicating results, and avoiding subtle pitfalls of confounder selection.³ Furthermore, they do not require parametric assumptions such as linearity.

A similar approach can be adopted to portray more complex relationships. The idea of a web of causation dates back to 1960⁸: an early diagram, reproduced here as Figure 1, attempted to explain why blood-born hepatitis was observed to be associated with syphilis.⁹ The authors of the diagram worked backward from this medically defined problem to examine its antecedents.

View larger version (23K): [in this window] [in a new window]   FIGURE 1— The web of causation as depicted by MacMahon and Pugh. Source. Reproduced with permission from Lippincott Williams & Wilkins.9

View larger version (35K): [in this window] [in a new window]   FIGURE 2— The Dahlgren and Whitehead schema of factors that influence health.10

We believe such complex systems should be mapped as networks of specific causal pathways to harness the strengths of epidemiology and integrate them with other methods of analysis. We will show how diagrams can aid such a theoretical synthesis by (1) specifying the causal linkages in complex systems, (2) organizing interdisciplinary research, and (3) serving as a basis for modeling. Another advantage is that because many people find it difficult to conceptualize complex systems unless clear means are devised to depict them,^14,15 even simple diagrams can encourage a focus on multistrand pathways, whereas traditional epidemiology tends to be limited to a single causal strand or even to just one link.

(Feedback loops, a feature of some diagrams, introduce analytic considerations that are beyond the scope of this article. The quantitative and statistical aspects of diagrams and their associated models, such as estimation of path coefficients, use of sensitivity analysis to explore assumptions, and use of simulation to investigate system dynamics, also are beyond the scope of this article.)

	DIAGRAMMING COMPLEX CAUSAL PATHWAYS

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 
Outside the context of communicable diseases, diagramming complex systems is comparatively undeveloped in the fields of epidemiology and public health. Patz et al. used a similar diagrammatic approach to show the potential impacts of climate change on a range of health outcomes through regional weather changes and other intermediate stages (e.g., air pollution levels and contamination pathways).¹⁶ The World Health Organization has developed the DPSEEA (Driving force, Pressure, State, Exposure, Effect, Action) diagrammatic model.¹⁷ Rehfuess applied a similar diagram to respiratory diseases that grouped factors into context, exposure, health outcome, and action.¹⁸ Marmot used diagrams to illustrate social-structural, life-course, psychosocial, and neuro-endocrine-immune influences on health.¹⁹ These authors used diagrams to organize knowledge, but they did not include statistical analysis or modeling, and they did not focus on change models (see "Change Models" section).

A diagram has been used to illustrate the various pathways through which traffic volume and speed affect a range of health outcomes²⁰; the air pollution part of this was modeled to estimate the potential health gain for a local government area in central London should the United Kingdom achieve its national air quality target.²¹ Bicego and Boerma used a diagrammatic conceptual framework to organize the statistical analysis of the relationship between maternal education, key confounders, intermediate variables, and infant mortality in a developing country context.²² Murray et al. have examined aspects of the statistical analysis of such structural systems.²³

How then can diagramming best be used to make sense of the causal complexities underlying major noncommunicable diseases and other health problems? First, we need to navigate between the extreme specificity of Figure 1 and the generality of Figure 2. Among the many possible choices between breadth and specificity, and the possible principles that could be used to structure the problem, we favor dividing the global scope of Figure 2 into manageable chunks according to economic sector rather than by, for example, clinical categories or major risk factors. Each policy area can then be further divided into submodels as required, which makes it possible to replace the overarching semicircles (or the Sussers’ Chinese boxes) with clearly specified causal pathways.

Figure 3a shows the links between transport and health. Epidemiology quantifies the causal relationships that end in a health outcome, such as cardiovascular disease affected by physical activity, respiratory symptoms affected by air pollution, and deaths/injuries that result from traffic collisions; complementary methods are required to fill in the other links in the upper part of the diagram. However, it is not just a question of adding transport expertise to health expertise; the links in the upper part of the diagram gain their importance from their relationship to health. In this sense, the diagram is constructed upwards—everything is driven by the bottom line.

View larger version (31K): [in this window] [in a new window]   FIGURE 3— Suggested diagrams of complex causal systems, (a) linking health outcomes to transportation, (b) linking changes in health outcomes to transport policies, and (c) showing the predicted health impact of controlling traffic speed. Note. As a convention, the health outcomes are shown as being negative (harmful). This can be criticized as embodying a medical model of health and ignoring positive health, but that is not the intention. For example, if mental health is likely to improve as a result of an increase in physical activity, the chart registers this in terms of a reduction in impaired mental health (without use of clinical categories). There are 2 reasons for adopting this convention. First, it helps with reading the charts if the outcome always carries the same type of implication. Second, while it would have been possible to introduce a positive convention instead, in practice most recorded health outcomes are problems—deaths, diseases, injuries—and arguably these also are more likely to influence policy makers than more general considerations of well-being and positive health, however important we may consider these to be. The causal direction for each link is specified with directional arrows. We use "+" for an increasing function (more of the item in the source box leads to more of that in the destination box), "–" for a decreasing function, and no sign indicator if mixed. If color is available, we prefer to use a color code: blue, red, and black, respectively. For any particular chain of causation from a specific policy intervention to a specific health outcome, the overall impact is positive (health gain) if there is an odd number of "–" (red) arrows, and negative (harmful) if there is an even number; however, the presence of a mixed (black) arrow makes the overall effect of the chain indeterminate. For this reason, it is best to avoid arrows of mixed sign, as with a hump-backed function that has rising and falling elements, and replace them with separate arrows that have unambiguous polarity.2 No attempt has yet been made in these charts to quantify the relationships shown. The thickness of the arrows can be used to show the strength of the causal association. The nature of the line (e.g., continuous, dashed, or dotted) can show the degree of confidence in the judgment that the causal link exists (i.e., that it is different from zero, so that the null hypothesis is rejected, which is approximately equivalent to a P value in statistics). The length can be used to represent duration, because causal relationships require time to take effect (e.g., latency in the causation of a disease such as cancer); in practice, this makes drawing diagrams complicated, and an alternative method is to place a delay box within the arrow.2 Finally, it would be simple to create an electronic version that enables the reader to click from an arrow to text that provides evidence for its existence, strength, etc., and back again.

Diagrams are rarely confined to the details of a single instance; rather, they seek to represent the broad class of systems to which a given instance belongs. For example, a flow chart of the processes that occur when diagnosing pediatric cardiac anomalies is a system in this sense, because each child is a realization of the system.²⁸ Clearly, it is necessary to carefully justify the assumption that the system is invariant in different circumstances.

	CHANGE MODELS

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 
So far, this discussion has been based on a levels model, which is the usual framework and is shown in Figures 1, 2, and 3a. However, a change model has many advantages: (1) it is more tractable, because only the elements that alter (including effect modification) need to be considered; (2) it readily connects with policies and other initiatives that could affect health; and (3) it has a clearer interpretation that can readily be understood, e.g., the health impact of a factory closure,²⁹ whereas inferring causality from a comparison of employed and unemployed people is fraught with difficulty.³⁰

The starting point is familiar to epidemiologists: when considering practical public health issues, the concern is mainly with changeable risk factors and how their alteration leads to a change in health status. Moving upstream to the determinants of the health determinants, changes fall into 3 categories: (1) spontaneous natural changes or societal trends, (2) policy-induced changes, and (3) the construction of scenarios for use in modeling.³¹ The first includes weather fluctuations, climate change, and population growth or an aging population. It also includes economic growth that has effects on health, both directly through increased prosperity and indirectly through increased road traffic and meat consumption.³² Policy-induced changes can be included within a model, but it is preferable to treat them as exogenous and to document the health impacts of the policy options.²⁰ Scenarios have been underused in the field of epidemiology; according to McMichael, "We must also address the issue of sustainability [now that human activity is able] to change the conditions for life on Earth,"^13(p894) rather than remain reactive.

For diagrams representing more specific topics, such as Figure 3a, the focus is now on the change from one period to the next rather than on a static system. Change models are particularly useful when analyzing the health impacts of policy options.²⁰ Thus, Figure 3a is redrawn as Figure 3b to show the health impact of transport policies, with the top line changing to indicate a series of exogenously given types of policy, and subsequent lines showing consequent changes in health determinants and health outcomes (indicated by ).

	METHODOLOGICAL ISSUES

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 
Diagrams share many of the functions of models, including making assumptions explicit, providing a framework for data analysis, generating testable predictions and projections, and exploring the effects of interventions or the introduction of a new technology—all of which are routine in communicable disease epidemiology. By specifying the complete causal system, diagrams help identify data gaps or weak links. Additionally, they can readily be combined with an economic evaluation of the outcomes.

Diagrams that relate health outcomes and health determinants to their upstream causes involve a mixing of languages, with the lower part being primarily biological and the upper part being more concerned with social and economic processes, thus providing a framework to integrate different disciplines, e.g., transport and health. They also facilitate the engagement of stakeholders (people involved in or affected by a proposal), for example, in the context of health impact assessment. Issues that must be considered are (1) the possible biases introduced when incorporating causal knowledge from a static study into a change model, and (2) the inclusion of different units of analysis, such as individual level in the lower part of the diagram and population or locality level in the higher part.^13,33–36

There is no guarantee that a unique diagram exists for any particular situation—different experts may construct the causal relationships differently. While at first sight this is a disadvantage, it is in fact a strength of the proposed approach: competing diagrams can be compared, which is one way of setting the agenda for empirical research, and ultimately the research findings will lead to the most appropriate diagram being selected.

Empirical Data
Ideally, causal diagrams are constructed on the basis of good empirical information—a combination of data on individual links, black box studies (i.e., those that show association without specifying the intervening links), and more complex investigations. Sometimes it may be necessary to infer the magnitude of an individual link indirectly from the model as a whole or from a submodel, which is the case with the transmission rate of microorganisms.²⁴ Obtaining this information requires appropriate study designs.³⁷ A useful tactic is to exploit policy interventions as natural experiments.^38–40 Typically, the evidence is inadequate (or even absent) for some of the specified links. It is still possible to construct a diagram, but its status is far less secure than a diagram that is evidence based. The process as we conceive it begins with a diagram that is plausible in the light of current evidence, and therefore relies substantially on judgment and generates the necessary empirical research, which leads to a well-supported diagram that gives an accurate representation of the complex system. Follow-up monitoring can be undertaken after policy implementation, and the results can be compared with the model. Furthermore, the diagram can be used for policy reevaluation.

To include a box or an arrow does not imply that they necessarily have the effect depicted; for example, if research were to demonstrate that a particular link does not apply (i.e., its magnitude is zero), then in the future that arrow would be deleted from the diagram. Study design should aim to distinguish findings that are clearly negative from those that are merely equivocal (i.e., falsifiability). A more serious potential problem when creating a diagram is the decision to omit a particular pathway when it should be included, because that pathway is likely to be ignored in subsequent empirical testing and no self-correcting mechanism exists to put it back on the agenda.

Structural Aspects
The structure of a diagram specifies independent chains of causation. In Figure 3b, one causal chain shows how emissions control policies affect changes in respiratory and cardiovascular morbidity and mortality, another chain demonstrates how promotion of active transport (i.e., walking and bicycling) improves various health outcomes via increased physical activity and access, and other chains show how different types of policy affect road deaths and injuries. These chains need to correspond to independent pathways. One implication is that joint action needs to be separately considered, for example, the use of sticks as well as carrots, the combination of which may be more effective (supra-additive) than the sum of their effects when used alone.

Effect modification must be considered, and can be represented as an arrow from a variable (the effect modifier) to another arrow (the association that is modified) rather than to another variable.²³ Additivity is another issue: in a complex causal system, the effects of separate but correlated factors are not in general additive, because multiple interacting risk factors act simultaneously. For example, infant mortality owing to a combination of malnutrition and indoor air pollution can be reduced by removing either factor, but the sum of both individual effects is not the same as the effect of removing both.²³ A change model makes a specific focus on the removal or alteration of one factor possible, which may be easier to interpret.

Human Agency
One question is how to handle the issue of human intervention, or agency. In principle, agency can be included in scientific studies, which is routine in psychology. It can involve modeling the policy decision process itself, not just the health impacts of different options. An alternative is to model the optimal decision in the tradition of decision theory,³⁶ either for use with decision support in a nonprescriptive manner or to specify the "rational" course of action in light of inbuilt assumptions, data, and structural/institutional constraints.

These methodologies can be characterized as analysis of policy. A third possibility is to treat decisionmaking as external to the model—analysis for policy (e.g., health impact assessment). The most complete way is to examine the range of health effects for each possible policy option in the context of a comparison of options.²⁰ This lends itself to a division of labor between the technical process of analyzing health consequences and the political process of policy development and decisionmaking, which involves taking into account underlying values, many different types of outcome, tradeoffs between positive and negative aspects, and lines of accountability.²⁰ However, the choice of the range of policy options cannot be regarded as merely neutral and apolitical, because failure to consider those that are regarded as controversial, possibly as a result of pressure from those who have economic or political vested interests, is itself a political choice. The course of action must be to examine all possible options.

Feedback
Until this point, we have been assuming that feedback and 2-way causation (which would be represented by double-headed arrows in a diagram) are absent. The examples in Figure 3 show that this is a reasonable assumption for many purposes, especially if the aim is to analyze the health impacts of policy options treated as exogenous variables. This also holds for many other examples of conjectural diagrams that apply, for example, to food/nutrition and transport.⁴¹ When this assumption cannot be made, it is necessary to pay explicit attention to how the variables involved in such a relationship mutually influence one another over time, which introduces additional analytic considerations. This is likely to be more of a problem when behavioral antecedents are included (e.g., the mutual dependence of food intake and nutritional status)⁴²—and especially if the policy decision itself is included within the model—than when the more structural factors we have discussed are included.

However, in many circumstances the possibility of risk compensation—a behavioral response to an environmental exposure—must be considered. For example, if a dangerous bend in the road is straightened, drivers may respond by increasing their speed,⁴³ a form of negative feedback that reduces the impact of the intervention. From an analytical viewpoint, it reduces the magnitude of the link (possibly to zero) in the same way that measurement error does. Risk compensation can also take a more complex form. For example, the impact of traffic on child health may include not only road injuries but also a more inactive indoor lifestyle that may be partly induced by parental fear of possible injury.^44,45 The compensatory action has far-reaching implications for public health, and in such cases, a more complex diagram is required.

As stated in an earlier section, modeling of systems that contain feedback loops is a well-developed and complex area that is beyond the scope of this article. When present, feedback strongly influences the dynamic behavior of the system, and a modification of conditions can produce complex changes that require simulation to explore them.^1,2,46

Instrumental Variables
Change models that are created on the basis of exogenous interventions can be used to control confounding and reverse causation by introducing an upstream influence that is independent of the putative causal influence(s). A similar approach—the use of instrumental variables—is routinely used in econometrics^47,48 and is the observational equivalent of intention to treat analysis in the context of randomized controlled trials. It assumes the absence of alternative pathways and effect modification. Instrumental variables have not traditionally been used much in epidemiology,^42,49 although a specific instance—mendelian randomization—has been discussed.⁵⁰ This requires additional biological assumptions.⁵⁰ Thus, epidemiological studies of social-level interventions used as natural experiments^38–40 have some methodological advantages over purely individual-level observational studies.

	PRACTICAL ISSUES

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 
With a policy focus, the division of material according to economic sectors is particularly appropriate, because these sectors generally correspond to the different government departments (or the equivalent at the local level). Any policy intervention has a range of possible outcomes—foreseen and unforeseen—and health is just one consideration. For example, interventions for reducing smoking must consider employment implications.⁵¹

Consistency also is required when determining what types of entry should go in each row, without being overproscriptive. Figure 1 has a row with treatment in clinics, economy, lack of knowledge, and human frailty, which as a group is highly heterogeneous and seems to represent a mental map rather than an attempt to map out actual causal relations in the real world. It is better to be more systematic. For example, in Figure 3b, there are rows for policy interventions, health determinants and health outcomes; in Figure 3c an additional row was added, because it makes sense to include vehicle speed as a separate row. This disciplined flexibility is important to achieving the most appropriate structure.

Because both social exclusion and a gradient of social inequalities are important factors in the causation of health and ill health,⁵² these—in addition to overall health—can be included in diagrams. For example, the lower part of the diagram can show the association of risk for preterm delivery with level of the stress hormone corticotrophin releasing hormone (CRH), and the upper part can show the distribution of CRH levels associated with socioeconomic status and experience of racial discrimination.⁵³ The change version might examine the impact of a poverty-reduction program on both CRH levels and preterm delivery risk.

	CONCLUSION

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 
Methods are available for epidemiologists to use causal diagrams for describing and modeling complex systems. They have some strengths that traditional epidemiology lacks, including rigorous causal thinking (as we saw with confounding and selection effects), integration of disparate information, and introduction of exogenous instrumental variables. They also have the potential to answer calls for interdisciplinary analyses of the forces that influence health.^{8,11–13,27,34,35,53,54} The scientific basis of public health will benefit if epidemiologists and experts in complementary disciplines collaborate to adopt this broader methodology.

	Acknowledgments

 
This project was funded by the Department of Health for England. The authors would like to thank Philip Dawid and Nancy Cartwright for their comments on an earlier draft.

Human Participant Protection
No protocol approval was needed for this study.

	Footnotes

 
Peer Reviewed

Contributors
M. Joffe led the writing, and J. Mindell contributed key empirical examples and arguments. Both authors originated ideas and reviewed all drafts of the article.

Accepted for publication July 17, 2005.

	References

TOP ABSTRACT INTRODUCTION HOW DIAGRAMMATIC METHODS HAVE... DIAGRAMMING COMPLEX CAUSAL... CHANGE MODELS METHODOLOGICAL ISSUES PRACTICAL ISSUES CONCLUSION References

 
1. Richardson GP. Feedback Thought in Social Science and Systems Theory. Philadelphia, Pa: University of Pennsylvania Press; 1991.

2. Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, Mass: Irwin McGraw-Hill; 2000.

3. Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology. 1999:10:37–48.[CrossRef][ISI][Medline]

4. Robins JM. Data, design, and background knowledge in etiologic inference. Epidemiology. 2001;11: 313–320.

5. Pearl J. Causality: Models, Reasoning and Inference. Cambridge, England: Cambridge University Press; 2000.

6. Hernán MA, Hernández-Díaz S, Werler MM, Mitchell AA. Causal knowledge as a prerequisite for confounding evaluation: an application to birth defects epidemiology. Am J Epidemiol. 2002,155:176–184.[Abstract/Free Full Text]

7. Meadows DH. The unavoidable a priori. In: Randers J, ed. Elements of the System Dynamics Method. Cambridge, Mass: MIT Press; 1980:23–57.

8. Krieger N. Epidemiology and the web of causation: has anyone seen the spider? Soc Sci Med. 1994;39: 887–903.[CrossRef][ISI][Medline]

9. MacMahon B, Pugh TF. Epidemiology Principles and Methods. Boston, Mass: Little Brown and Co; 1970.

10. Dahlgren G, Whitehead M. Policies and Strategies to Promote Social Equity in Health. Stockholm, Sweden: Institute for Future Studies; 1991.

11. Susser M, Susser E. Choosing a future for epidemiology: II. From black box to Chinese boxes and eco-epidemiology. Am J Public Health. 1996; 86:674–677.[Abstract/Free Full Text]

12. Shy CM. The failure of academic epidemiology: witness for the prosecution. Am J Epidemiol. 1997;145: 479–487.[Abstract/Free Full Text]

13. McMichael AJ. Prisoners of the proximate: loosening the constraints on epidemiology in an age of change. Am J Epidemiol. 1999;149:887–897.[Abstract/Free Full Text]

14. Dörner D. On the difficulties people have in dealing with complexity. Simulation Gaming. 1980;11:87–106.[CrossRef]

15. Dörner D. The Logic of Failure. New York, NY: Metropolitan Books/Henry Holt; 1996.

16. Patz JA, McGeehin MA, Bernard SM, et al. The potential health impacts of climate variability and change for the United States: executive summary of the report of the health sector of the US National Assessment. Environ Health Perspect. 2000;108:367–376.[ISI][Medline]

17. Briggs DJ, Wills J. Presenting decision-makers with their choices: environment health indicators for NEHAPs. In: Briggs D, Stern R, Tinker TL, eds. Environmental Health for All. Dordrecht, the Netherlands: Kluwer Academic Publishers; 1997:187–201.

18. Rehfuess E. WHO/EEA joint workshop on "children’s environment and health indicators". Copenhagen, WHO, 2002. Available at: http://www.euro.who.int/document/E80604.pdf. Accessed October 5, 2005.

19. Marmot M. Multilevel approaches to understanding social determinants. In: Berkman LF, Kawachi I, eds. Social Epidemiology. New York, NY: Oxford University Press; 2000:349–367.

20. Joffe M, Mindell J. A framework for the evidence base to support Health Impact Assessment. J Epidemiol Community Health. 2002;56:132–138.[Abstract/Free Full Text]

21. Joffe M, Mindell J. Predicted health impacts of urban air quality management. J Epidemiol Community Health. 2004;58:103–113.[Abstract/Free Full Text]

22. Bicego GT, Boerma JT. Maternal education and child survival: a comparative study of survey data from 17 countries. Soc Sci Med. 1993;36: 1207–1227.[CrossRef][ISI][Medline]

23. Murray CJL, Ezzati M, Lopez AD, Rodgers A, Vander Hoorn S. Comparative quantification of health risks: conceptual framework and methodological issues. Popul Health Metrics. 2003;1: 1–20.

24. Anderson RM, May RM. Infectious Disease of Humans: Dynamics and Control. Oxford, England: Oxford University Press; 1991.

25. Ghani AC, Ferguson NM, Donnelly CA, Anderson RM. Factors determining the pattern of the variant Creutzfeldt-Jacob disease (vCJD) epidemic in the UK. Proc Biol Sci. 2003;270:689–698.

26. Riley S, Fraser C, Donnelly CA, et al. Transmission dynamics of the etiological agent of SARS in Hong Kong: impact of public health interventions. Science. 2003;300:1961–1966.[Abstract/Free Full Text]

27. Savitz DA, Poole C, Miller WC. Reassessing the role of epidemiology in public health. Am J Public Health. 1999; 89:1158–1161.[Abstract/Free Full Text]

28. Spiegelhalter DJ, Dawid AP, Lauritzen SL, Cowell RG. Bayesian analysis in expert systems. Stat Sci. 1993;8:219–283.

29. Keefe V, Reid P, Ormsby C, et al. Serious health events following involuntary job loss in New Zealand meat processing workers. Int J Epidemiol. 2002; 31:1155–1161.[Abstract/Free Full Text]

30. Bartley M, Ferrie J, Montgomery SM. Health and labour market disadvantage: unemployment, non-employment, and job insecurity. In: Marmot M, Wilkinson RG, eds. Social Determinants of Health. 2nd ed.Oxford, England: Oxford University Press; 1999:78–96.

31. Ghani AC, Ferguson NM, Donnelly CA, Hagenaars TJ, Anderson RM. Epidemiological determinants of the pattern and magnitude of the vCJD epidemic in Great Britain. Proc Biol Sci. 1998;265:2443–2452.

32. The World Bank. World Development Report 1993. New York, NY: Oxford University Press; 1993.

33. Rose G. The Strategy of Preventive Medicine. Oxford, England: Oxford University Press; 1992.

34. Kaufman JS, Cooper RS. Seeking causal explanations in social epidemiology. Am J Epidemiol. 1999;150: 113–120.[Abstract/Free Full Text]

35. Pearce N. Traditional epidemiology, modern epidemiology, and public health. Am J Public Health. 1996;86: 678–683.[Abstract/Free Full Text]

36. Dowie J. The role of patients’ meta-preferences in the design and evaluation of decision support systems. Health Expect. 2002;5:16–27.[CrossRef][Medline]

37. Thomson H, Hoskins R, Petticrew M, et al. Evaluating the health effects of social interventions. BMJ. 2004;328: 282–285.[Free Full Text]

38. Wanless D. Securing Good Health for the Whole Population. London, England: HM Treasury; 2004.

39. Hedley AJ, Wong C-M, Thach TQ, Ma S, Lam T-H, Anderson HR. Cardiorespiratory and all-cause mortality after restrictions on sulphur content of fuel in Hong Kong: an intervention study. Lancet. 2002;360: 1646–1652.[CrossRef][ISI][Medline]

40. Clancy L, Goodman P, Sinclair H, Dockery DW. Effect of air-pollution control on death rates in Dublin, Ireland: an intervention study. Lancet. 2002;360:1210–1214.[CrossRef][ISI][Medline]

41. Wellbeing, Health and Economic Policy Services. "Mapping the links" project. Available at: http://www.wheps.org/mapping.htm. Accessed December 19, 2005.

42. Zohoori N, Savitz DA. Econometric approaches to epidemiologic data: relating endogeneity and unobserved heterogeneity to confounding. Ann Epidemiol. 1997;7:251–257.[CrossRef][ISI][Medline]

43. Adams J. Risk. London, England: UCL Press; 1995.

44. Hillman M, Adams J, Whitelegg J. One False Move ...: a Study of Children’s Independent Mobility. London, England: Policy Studies Institute; 1990.

45. Adams J. Evaluating the effectiveness of road safety measures. Available at: http://www.geog.ucl.ac.uk/~jadams/pdfs/evaluating%20safety%20measures.pdf. Accessed October 5, 2005.

46. Homer J, Oliva R. Maps and models in system dynamics: a response to Coyle. System Dynamics Rev. 2001;17: 347–355.[CrossRef]

47. Kennedy P. A Guide to Econometrics. 4th ed. Oxford, England: Blackwell Publishers Ltd; 1998.

48. Greene WH. Econometric Analysis. 4th ed. Upper Saddle River, NJ: Prentice-Hall Inc; 2000.

49. Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol. 2000;29: 722–729.[Abstract/Free Full Text]

50. Davey Smith G, Ebrahim S. "Mendelian randomization": can genetic epidemiology contribute to understanding environmental determinants of disease? Int J Epidemiol. 2003;32:1–22.[Abstract/Free Full Text]

51. Buck D, Godfrey C, Raw M, Sutton M. Tobacco and Jobs. York, England: Centre for Health Economics, University of York; 1995.

52. Shaw M, Dorling D, Davey Smith G. Poverty, social exclusion, and minorities. In: Marmot M, Wilkinson RG, eds. Social Determinants of Health. 2nd ed. Oxford, England: Oxford University Press; 1999:196–223.

53. Krieger N. Sticky webs, hungry spiders, buzzing flies, and fractal metaphors: on the misleading juxtaposition of "risk factor" versus "social" epidemiology. J Epidemiol Community Health. 1999;53:678–680.[ISI][Medline]

54. Diez-Roux AV. Bringing context back into epidemiology: variables and fallacies in multilevel analysis. Am J Public Health. 1998;88:216–222.[Abstract/Free Full Text]

This Article Abstract Figures Only Full Text (PDF) All Versions of this Article: AJPH.2005.063693v1 96/3/473    most recent Submit a response Purchase Article View Shopping Cart Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Joffe, M. Articles by Mindell, J. Search for Related Content PubMed PubMed Citation Articles by Joffe, M. Articles by Mindell, J. Related Collections Epidemiology Health Policy Prevention Socioeconomic Factors