• logo linkedin
  • logo email
Visuel
This evaluation assesses three projects aiming at similar objectives in very different contexts. Its goal was to learn overall lessons after having worked with three NGOs, from the start of their interventions and based on a harmonized monitoring system.
Context

The Sectoral Innovation Facility for NGOs (FISONG) is an AFD financing tool that helps enhance specific know-how of NGOs and their capacity for innovation. A call for projects was launched in 2012 on the theme of “Biodiversity and Development: sharing the benefits of biodiversity to help village communities.” In early 2013, three NGO projects were selected via a call for projects in Tanzania, Niger and Madagascar.

Goals

Firstly, at the start-up phase of each project, the evaluation sought to determine a monitoring-evaluation system that would enable comparison of the projects on certain key indicators. Secondly, it sought to perform an evaluative analysis of three projects while conducting cross-cutting capitalization work that included a workshop to discuss the outcomes and recommendations with the NGO beneficiaries.

Method

The consulting firm Oréade-Brèche carried out this evaluation and capitalization. It tried to harmonize a monitoring system targeting the three projects, which were designed entirely independently from one another in totally different contexts and with distinct intervention approaches. It moreover tried to improve, as much as possible (and without additional cost to the budget allocated to the NGOs), the logical frameworks of the projects initially proposed, and thus the final evaluability of the three interventions. The qualitative evaluations involved a desk review produced by the monitoring systems adopted by each NGO, rounded out by interviews and a field mission in each country. Based on the three evaluation reports produced, the consulting firm submitted a capitalization report comparing the three projects; it used a single set of 26 technical criteria on governance, capacity building, conservation of biodiversity, socioeconomic development, and sustainability of gains made in protected areas.

Results

While relevance was satisfactory overall, effectiveness and efficiency were less so. As for the sustainability and impacts of these projects, their evaluation remains difficult, with outcomes variable depending on the sites. The project in the Saharan zone of Niger obviously suffered from conditions of insecurity, but it did manage to achieve certain outcomes, though with high management costs and uncertain sustainability. In Madagascar, the intervention was positive in many aspects but remained strongly influenced by poor public governance. The project in Tanzania seems to have reached a greater number of its governance and development objectives, but with impacts on biodiversity conservation difficult to evaluate.

This combined exercise of evaluation and capitalization itself lasted more than two years. Its many advantages were acknowledged by the participants, and the additional cost to the evaluations (around 25% more than the total of the three evaluations) was reasonable. The advantages were:

  • the development of a more robust monitoring method than the logical frameworks of each project, thanks to the contributions in methodology by the consulting firm and the workshop discussions;
  • regular exchanges among the project initiators on technical themes for which exchanges among NGOs are few in number outside of their own internal circles;
  • and the final discussion on the recommendations, which included contributions from each party and made it possible to put forward specific and more generic recommendations.
Teachings

In light of the advantages of the approach, the evaluators recommended that it be replicated by AFD more systematically. Many recommendations stemming from the summary of the evaluations of the three projects concerned improvement in the content and monitoring of projects to support protected areas carried out by conservation NGOs. Among other things, it was proposed that the initial state (in socioeconomic and biodiversity terms) of the sites be subject to closer attention during project preparation, as the projects can be improved through greater utilization of the statistical data available for free on the Internet.

The consultants observed a recurring problem when support projects for protected areas are prepared. This problem involves conveying the message, in the presentation document, that tangible outcomes in conservation and development are going to be achieved after a short period of project implementation, without the initial conditions of the protected area necessarily having been taking into account. Without such “exagerated” ambitions from the start, the NGO initiating the project runs the risk of not seeing its project selected by the call-for-projects system. Yet, the final evaluations showed, nearly systematically, that the ambitious outcomes aimed for in the beginning were in the end not achieved.

The consultants recommend adopting a more reasonable and effective approach. They propose a graphical tool to be used with “radar” effect, seeking to describe (i) the initial state of the protected area (and not of the project) and (ii), using this same graph, the outcomes of the progress the protected area claims to achieve at the end of the project. This approach would make it possible both to avoid having to observe that, in the end, few protected area projects achieve the outcomes that had been set for them in the beginning, and to show (if still needed) that these processes are long and must go through successive stages. The approach helps to clarify that reaching the outcomes of these different phases takes much time (15 years according to another capitalization of this type, in French) and conclusively more than the period of a single traditional project of two to four years, while at the same time it makes it possible to measure progress and ensure accountability to donors

15/04/2014
Project start date
26/01/2017
Project end date
2 years and 9 months
Duration of the program
Sectors
Madagascar, Niger, Tanzania
Location
Achevé
Status

Contact : Julien Calas, evaluation officer (calasj@afd.fr)