The Limpopo National Park (LNP) project evaluation was unusual in that it was both written and filmed. AFD was counting on the fact that the film images, interviews and animated maps would give a less austere rendering of the goals of the funded development, the physical environment and wildlife, the villages’ economic development and the concrete results achieved.
As part of a learning objective, this approach aimed to turn the evaluation into a fresh opportunity for dialogue among the project stakeholders and, further afield, to reach development professionals, especially those interested in biodiversity-related issues.
Several difficulties arose in this “mixed” evaluation. The first involved ensuring a methodological coherence between the two evaluations (written and filmed) as it needed to lead to the same evaluative conclusion. The solution devised was to co-develop the evaluation methodology and systematically ensure an exchange of viewpoints at every stage of this method.
It was decided at the outset that the written and filmed evaluations would take place simultaneously and follow exactly the same construction and validation steps. As a result, just one purchase order was issued to recruit an integratedteam.
Further reading: Rehabilitating Limpopo National Park
One difficulty encountered during data collection: the challenge of fully coordinating the two evaluations carried out by an integrated team that had distinct specialties (written and filmed), in the same field and with the same actors. It was decided that joint interviews would be held, especially during framework interviews, to gather general information on the issues under discussion and the interviewee’s role in the project.
However, the written evaluation interviews needed to go into greater detail than was relevant for the filmed evaluation. Thus, to limit the number of interviews and boost efficiency, the teams divided up the interviews depending on the role of the project stakeholder interviewed.
A complementary view of the projects results
The LNP filmed evaluation confirms that this filmed approach shows the practical aspects, the dynamics and context of the project but is ill-suited to analysing organisational, financial and methodological processes. It gives value to stakeholders’ (beneficiaries or not) contextualised accounts and actual words. In short, the informative value of images is more powerful but less precise than the written word.
The evaluative film is more than a written report – it helps to raise awareness and encourage debate. This makes it an effective way of training development practitioners. It is also an opportunity to educate a wider audience about development challenges
The approach based on a concomitant written and filmed evaluation of the Limpopo project reinforced the evaluative analysis thanks to the exchange of the team’s different viewpoints. The filmed interviews were used to validate the secondary data collected and thus refine the analyses of the stakeholders’ interactions as well as the challenges and policies related to the project. The film more easily revealed the various or even opposing perspectives of some stakeholders.
These divergences were often less obvious in the written report, mainly as this calls on the intermediation of an author. The film also influences the narrative structure as it gives more detail on the history of the project and its challenges, whereas the written evaluation presents an analysis based on pre-defined evaluative criteria. Although the film does not replace the detailed sections on the institutional, budgetary, political and technical aspects of the written evaluation, it completes and strengthens the messages, notably for the project stakeholders.
An effective tool for awareness-raising and dialogue
At the feedback sessions on the conclusions of the LNP evaluation, the film – which explores the highly complex topic of national park protection together with its multiple stakeholders – made it easier to share the results and encourage debate, especially among the communities and staff in charge of protecting LNP. Misunderstandings were resolved and the dialogue on the LNP’s future development has continued. Finally, the filmed evaluation also helped to draw the public’s attention to its messages – a public that extends beyond the project stakeholders to include, for example, students or citizens interested in development. In short, a broader and more varied audience.
The use of the audiovisual format basically depends on what the sponsors want to do with it. What objectives did the evaluation set? Are they looking for the most in-depth evaluation possible or would they prefer a study of the salient points and dynamics of the project?
To continue the filmed evaluations
What are the prospects for filmed evaluations? The complementarity of written and audiovisual formats could be further developed in order to use the specific qualities of both words and images in the evaluative approach. Using the medium (written or filmed) that is best suited to understanding and evaluating each project component would provide a fairer and more relevant perception.
This approach could be formulated as a specific methodological reference that would help to frame the contexts in which a filmed evaluation would be much more relevant and clarify the methodology combining both formats. The AFD teams will test this in 2019 with new projects under development.
Before the Limpopo National Park evaluation, AFD had funded three filmed evaluations:
- Building against the Tides (2008), a filmed evaluation of the Prey Nup hydro-agricultural project in Cambodia, made a few years after the written evaluation. Given AFD's long history in the country, the objective was to “build on experience” more than evaluate the project results already covered by the written evaluation.
- Water Project in Soweto (2011): a filmed evaluation that is interesting due to its strictly audiovisual approach, but no less rigorous than a written evaluation. It followed the five evaluation criteria from the OECD Development Assistance Committee. Based on accounts and interviews, this evaluation has not only captured the interplay and interactions among project stakeholders, but has gone further as it shows them in a broader institutional context, particularly regarding the social aspects.
- Palestinian Municipalities: Looking for the Future (2012), a filmed evaluation funded by local partners, which rounds off a prior written evaluation. The evaluation focused on a support project for Palestinian municipalities in the areas of investment and governance. The objective was to more deeply explore one of the key challenges identified by the written evaluation and relating to the sustainability of the project. The goal was to gain better knowledge and analysis of the processes suppor ting the institutional organisation and the roles played by the different stakeholders. The local partner was closely involved in much of the preparatory work for this filmed evaluation.