Mercy Corps - early leader in evaluating sustainability... and what donors are funding
Mercy Corps shared their work in post-project sustainability early on, inspiring me it was possible. As they put it, "clearly, a sustained ability for collective problem solving offers the best path to lasting improvement in people's lives and, for donors, the best return on investment" in two conflict resolution projects in Kyrgyztan, Tajikistan and Uzbekistan: Peaceful Communities Initiative $6.5 mil (2002-2007) and Community Action Investment Program $11.8 mil (2002-2005). These two projects were complex community mobilization programs with aims "to empower communities to work together in a participatory manner to address the infrastructure and social needs [while] developing sustainable skills in problem solving, consensus-building and accountability. The process also empowers communities to begin to identify and utilize existing resources within the communities and not to depend only on external assistance."
So what happened? Their report on sustainability in 2007 random-sampled and interviewed youth leaders and in community action groups, 55% of the communities and found promising results:
According to the evaluation:
* 93% of surveyed projects are still being actively used by the community after programs closed.
* 73% of members of the community action groups (CAG) felt it was still easier to approach local government at least one year after the programs ended and 68% witnessed local government becoming more involved in community activities after the end of the programs as compared with before the programs... [participants and partners had] implemented almost 100 infrastructure projects by themselves and independent of donor funds.
* 72% of youth report that they continue to use at least one skill they learned during the programs. Those cited most often include teamwork and communication, as well as practical skills such as sewing, construction, roofing, journalism and cooking
* 57% of the communities studied continuing to use one or more of the decision-making practices promoted during the program.
* 42% of CAG members, representing 35 of the 51 communities, reported that the community had worked collectively on new projects or repairs to existing infrastructure
* In total, 40% of general community members interviewed reported that youth had initiated community activities since January 2007, and 68% of these community members recognized that some or all of the activities had not taken place prior to Mercy Corps’ program
* 93% of surveyed projects are still being actively used by the community after programs closed.
* 73% of members of the community action groups (CAG) felt it was still easier to approach local government at least one year after the programs ended and 68% witnessed local government becoming more involved in community activities after the end of the programs as compared with before the programs... [participants and partners had] implemented almost 100 infrastructure projects by themselves and independent of donor funds.
* 72% of youth report that they continue to use at least one skill they learned during the programs. Those cited most often include teamwork and communication, as well as practical skills such as sewing, construction, roofing, journalism and cooking
* 57% of the communities studied continuing to use one or more of the decision-making practices promoted during the program.
* 42% of CAG members, representing 35 of the 51 communities, reported that the community had worked collectively on new projects or repairs to existing infrastructure
* In total, 40% of general community members interviewed reported that youth had initiated community activities since January 2007, and 68% of these community members recognized that some or all of the activities had not taken place prior to Mercy Corps’ program
This appears excellent! They interviewed youth and key local stakeholders (CAG), used country nationals to evaluate (both of which are wonderful as it is their country) and they distilled best practices in community mobilization in accessible reports shared within the international non-profit world via the web. Quibbling only a little, a difficulty is that while the report states that they "Identified factors that influence sustainability, through both positive examples and non-sustained projects and practices" our ability to learn from what didn't work was hidden. Mercy Corps discussed what didn't work only by recommending what to do, rather than discuss the extent to which specific activities simply didn't work. While focusing on the positive is the best path forward, ideally MErcy Corps would have also shared what failed, possibly why and whether they had seen this elsewhere (thereby suggesting such activities may not be promising to replicate). Further, what would be valuable is to gauge roughly the percentage of the program value these successes represented versus those that did not work well. Sucha cost-effectiveness ratio would benefit our industry.
More specifically, Nancy Macpherson of Rockefeller Foundation states the Foundation is "committed to evaluation practices that are rigorous, innovative, inclusive of stakeholders’ voices, and appropriate to the contexts in which the foundation works." This is done by "integrating the views of developing-region evaluators" as well as:
* "strengthening developing country evaluation practice and ownership of results...
* developing innovative methods and approaches to evaluation and learning...
* the empowerment of people; and
* the effectiveness of development interventions by national governments and international partners and, increasingly, by non-state actors—foundations, philanthropists, and agencies that promote investing for impact."
The Bill and Melinda Gates Foundation mentions cost effectiveness
* "When evidence is needed to fill a knowledge gap or evaluate a significant policy decision. Evaluation can help to resolve uncertainty and determine the relative cost-effectiveness of different interventions, models, or approaches" (Gates) and
* "Both quantitative and qualitative data are relevant in evaluating processes, operations, cost effectiveness, key stakeholders’ perceptions, and enabling contextual factors" (Gates)
USAID seems not to have done any ex-post evaluation since a single one on a Philiiipines loan in 1980, whereas parts of the EU seem to be doing many more such evaluations in agriculture and rural development (2002-06), industrial technologies (2009-11) as recently as 2013 for ICT. Given the size of its portfolio of development assistance, $20.4 billion in development and humanitarian programs in fiscal 2014, learning about sustained impact seems imperative from a return on investment (ROI) perspecive.
Supporting national capacity and evaluating programs' return on investment is pivotal. In Mercy Corps/ USAID's funding case alone, this comes to over $18 million dollars. But Mercy Corps paid for this evaluation themselves, from private funds. If we are in the process of fostering country-led and eventually country-financed development, they need to know how much such investments get them - as we do.
Does anyone know of such great program learning that begins to teach about return on investment? Can you share?
No comments:
Post a Comment