The use of causal mechanisms in complex evaluations
Analysing causal mechanisms in evaluations helps us understand how and why complex development programmes work. This enables us to unpack the ‘black box’ and provide more robust findings about programme effectiveness. The objective of this methods research project is to reconcile ongoing discussions and synthesize experiences with causal mechanism approaches. Contact Johannes Schmitt to join the causal mechanisms discussion.
Particularly in complex settings, theory-based evaluation (TBE) approaches highlight the usefulness of examining causal mechanisms. At the core of working with causal mechanisms is the systematic analysis of basic interrelations between (elements of) an intervention and its expected outcomes. In formative evaluations, mechanisms can provide a better understanding of the way in which complex programmes are implemented. In summative evaluations, analysing causal mechanisms enables the evaluator to arrive at more credible statements on causal links between an intervention and the observed outcomes.
Despite consensus on their usefulness, various concepts and approaches dealing with causal mechanisms are being discussed in the community. Most TBE-scholars agree that causal mechanisms explain what is going on between the intervention and the outcome; generate the observed outcome; enable evaluators to disentangle the effects of an intervention and answer questions about how and why. However, the concept of causal mechanisms is being applied differently across evaluation approaches. Mechanisms are described as causal processes in process tracing, contribution analyses, and other forms of theory-of-change approaches. Applying a broad analytical focus, they describe the causal process between (one particular element of) a program and one or more outcomes. Zooming into different parts of the broader theory of change, however, evaluators may also focus on behavioural and psychological explanations. They are termed ‘social mechanisms’ and play a role in realistic evaluation and in quantitative approaches of structural equation modelling. What is more, the use of causal mechanisms is not confined to the TBE-approaches and considered causal pathways or treatment arms in experimental designs.
The objective of this evaluation-methods project is reconciling ongoing discussions and drawing together experiences with causal mechanism approaches. Become part of the mechanisms discussion, share your experience from working with causal mechanisms in complex evaluations and engage in the new working group on causal mechanisms.
Schmitt, J. (2018), "Testing causal mechanisms with Bayesian process-tracing: Strengthening explanatory power of case study evaluations", The Evaluator.
Schmitt, J. (2018), Schluss mit Schwarzen Boxen. Zur Arbeit mit Kausalmechanismen in Evaluierungen, DEval Policy Brief 10/2018, Deutsches Evaluierungsinstitut der Entwicklungszusammenarbeit (DEval), Bonn.