“It is a mistake to theorize before one has data. Insensibly, one begins to twist facts to suit theories instead of theories to suit facts.”
Many organizations, it would seem, are currently exploring how their organization-wide monitoring and evaluation systems can be strengthened. This requires a strategic and thoughtful process of identifying how data is currently used, how it should be used, and how you want to use it.
M&E systems are critical for results-oriented organizations: it helps identify promising programs and/or practices; identify unintended, positive and negative, results and effects; identify program weaknesses and enable managers to take action to rectify the problem; and facilitate a learning culture.
The World Bank has two terrific, comprehensive how-to guides for establishing M&E systems and ‘making them work.’ When taken together, the authors lay out a 10-step plan for establishing an M&E system from scratch and are composed of 12 components. For the sake of brevity these have been shortened and the focus will be on the 10-steps to establishing a system and common challenges.
Step One – Conduct a Readiness Assessment to determine both the capacity and, critically, willingness of the organization (particularly its leaders) to develop a results-based M&E system. Your assessment should identify key barriers, the presence and absence of champions who can rally others to support and engage the system.
Hot Resource! Evaluation Needs Assessment by Johanna Morariu
Step Two – Identify Outcomes to Monitor and Evaluate which will be the focus of the organization-wide M&E system. These will likely be derived from a Strategic Plan and will drive the allocation of resources in the M&E system.
Step Three – Develop Key Indicators to Monitor Outcomes so you can determine the degree to which you are achieving the desired organizational outcomes. Significant time should be devouted to determining the most appropriate and accurate indicators for your desired outcomes. This can be politically and methodologically challenging.
Step Four – Conduct a Baseline on Indicators so that you have an initial measurement to determine progress and success against.
Step Five – Plan for Improvements on your programs based on the initial baseline assessment. The initial baseline will have highlighted programmatic areas for improvement. It can be helpful to develop benchmarks towards the achievement of your desired outcomes. Remember, change is oft long-term, complex, dynamic and non-linear. Progress will be made in some areas, while regression occurs in others.
Step Six – Monitoring for Results involves the establishment of data collection, analysis and reporting systems; designating staff to oversee and manage the processes; developing quality control mechanisms; etc. “It is emphasized that, in constructing an M&E system, the challenges of ownership, management, maintenance, and credibility need to be addressed clearly.”
Step Seven – Evaluative Information to Support Decision Making focuses on how the monitoring and evaluation data is used to support managerial and programmatic decisions in the organization. “Analysis of program theory, evaluability assessments, process evaluations, outcome and impact evaluations, and evaluation syntheses are among the strategies discussed that can be employed in evaluating a results-based M&E system.
Step Eight – Analyze and Report Findings involves identifying what findings are reported to whom, at what interval and in what format. This step may require capacity development to enhance information production and their methodologies, analysis and reporting.
Step Nine – Use the Findings to inform critical decisions in a timely and appropriate manner. The key decision makers would have already been identified: this step involves getting the information to them so that it can be considered in their decision.
Step Ten – Sustain the M&E System through continued capacity development, staff and partner buy-in, appropriate resource allocation, etc. “Six criteria are seen as crucial to the sustainability of an M&E system: demand, structure, trustworthy and credible information, accountability, incentives, and capacity.”
Of course, any M&E system should produce actionable results. The Bill & Melinda Gates Foundation suggest three elements as a guide for actionable measurement to lead to informed decisions and actions: planned collection, analysis, and synthesis of data and experience; time devoted to reflection and development of insight; and, a willingness and ability to change. These elements together lead to informed and evidence based decisions and actions. In this week’s DM&E Tip (to be posted later this week), we will explore key elements of actionable measurement in an M&E system.
Setting up an effective M&E system is an inherently political task: the presence of rigorous data can change the systemic dynamics of institutional and individual relations and their connections. It is therefore essential to have a champion of the M&E system who is either able to affect key-decision makers or is a key-decision maker.
But the larger issue here is the wider culture of the professional practice of peacebuilding towards M&E. One of the common sayings we have in the Institutional Learning Team at Search for Common Ground is, “You can’t manage knowledge, but can you manage the environment in which it is generated, used and disseminated.” That is to say, while knowledge management strategies matter (to which M&E is a part), one cannot ignore the people element. Staff buy-in, and the presence of supportive capacities, skills and a learning culture, is critical for an effective and sustainable M&E system. There is more to M&E systems than just M&E.
Jonathan White is the Content Manager for the Learning Portal for DM&E for Peacebuilding at Search for Common Ground. Views expressed herein do not represent SFCG, the Learning Portal or its partners or affiliates.
 Marelize Gorgens and Jody Zall Kusek, Making Monitoring and Evaluation Systems Work: A Capacity Development Toolkit, World Bank, accessed 11 June 2012, Making Monitoring and Evaluation Systems Work: A Capacity Developme....
 Jody Zall Kusek and Ray C. Rist, Ten Steps to a Results-Based Monitoring and Evaluation System, World Bank, pp. 4-6, accessed 11 June 2012, http://dmeforpeace.org/sites/default/files/Making_M_and_E_Systems_W....
 Kusek and Rist, “Ten Steps,” p. 5.