There is an emerging discussion in international development on theories of change. On the one hand you have methodological discussions and debates in theory-based evaluation on the testability and evaluability of theories of change, and on the other an emphasis on program design and the role and qualities of a good theory of change. This DM&E Tip attempts to capture the key points of the discussions on both sides (perhaps duality is a simplification, but it will suffice for this purpose).

First, a note on the difference between the approaches of understanding the evaluability of a theory of change and a ‘good’ theory of change:

“A theory of change may be evaluable because the theory is clear, and the relevant data is available (evaluability). But as the program is implemented, or following its evaluation, it might be discovered that the theory of change was wrong, that people or institutions don’t work the way the theory expected them to do so. It was a ‘bad’ ToC. Alternatively, it is also possible that a ToC may turn out to be good, but the poor way it was initially expressed made it un-evaluable, until remedial changes were made (‘good’ and ‘bad’ ToCs).”1

Hot Resource! Reflective Peacebuilding: A Planning, Monitoring and Learning Toolkit by John Paul Lederach, Reina Neufeldt and Hal Culbertson

Theory of Change in Design

Quality in theory of change is, generally, seen as coming from the quality of the thinking in the theory of change process.2 Isabel Vogel, in her current draft report on the Review of the use of ‘Theory of Change’ in international development for DFID (draft 4 available here), suggests the following points based on in-depth interviews with key actors (which I have adapted to be more immediately relevant to peacebuilding):

• Were and to what extent were local actors and partners brought in to the participatory development of the theory?
• Is/to what extent is the theory clearly grounded in the context, informed by local knowledge and sensitive to context and conflict systems and their dynamics?
• Is/to what extent is the theory supported by an initial in-depth analysis and how is the theory being monitored and adapted in accordance with the data?

I would, in addition, humbly suggest the Reflecting on Peace Practice matrix is relevant here as well:

• Does the theory include individual/personal-sociopolitical change and key-more people interactions and linkages?

Hot Resource! Reflecting on Peace Practice Training Manual by CDA, Inc.

Theory of Change in Evaluation

Hot Resource! Criteria for Assessing the Evaluability of Theories of Change by Rick Davies

There is a difference between a ‘good’ theory of change and an evaluable theory of change. Rick Davies (http://mandenews.blogspot.com/), a well-known evaluative thinker, suggests the following criteria (simplified for this post) as a guide for determining the evaluability of a theory of change:

• Understandable – is the theory easy to understand? Do people interpret it in the same way?
• Verifiable – are events described in a way that can be verified?
• Testable – are there identifiable causal links/pathway between the stated events?
• Explained – are assumptions explicit in the theory, or do they still need to be surfaced?
• Complete – does the chain of events make a connection between the intervention/intervening actor with the intended beneficiaries?
• Justifiable – is there evidence to support the sequence of events?
• Plausible – if there is no evidence to support the sequence of events, is the sequence plausible?
• Owned – who developed the ToC and how were other viewpoints taken into account in the theory?
• Embedded – has the theory been operationalized in project/program implementation?

Clearly, there are tremendous similarities between these two approaches, and both are important: any theory of change should attempt to be as close to the reality of the change process (and in addition to relevancy and participatory development of the theory is therefore a ‘good’ theory) and at the same time that theory should also be evaluable – otherwise, how will its accuracy, relevancy, etc. be tested, thus confirming that theory is, indeed, ‘good’?

As peacebuilders continue to explore the use and application of theory of change in our work, it is essential to keep these two (perhaps even more?) perspectives in mind.

Hot Resources

Criteria for Assessing the Evaluability of Theories of Change by Rick Davies

M and E News Blog by Rick Davies

Reflecting on Peace Practice Training Manual by CDA, Inc.

Reflective Peacebuilding: A Planning, Monitoring and Learning Toolkit by John Paul Lederach, Reina Neufeldt and Hal Culbertson