As many other commentators have noted, extension services in SA are in a very poor state.

 There is agreement on the symptoms (poor management, lack of client focus, no accountability to farmers, a ‘top down’ focus on ‘dispensing’ services rather than empowering farmers, lack of an integrated and collaborative approach and more. And many good ideas are proposed on this page to improve the system. But what is really going to drive this change? 

At Keystone Accountability (a new non-profit initiative based in Cape Town and London) we believe that the way we define success and the way we measure and manage our performance is often what really drives practice. For the past five years we have been working with thought-leading development funders and agencies to design more developmental performance management measures that give voice to and empower those intended to benefit. We call this approach Constituent Voice. CV is an empirically rigorous and systematic way of listening to and learning from our most important constituents – those in whose name we do our work – and then using this knowledge in dialogue with constituents to improve the relationships and the effectiveness of programs.

We have just begun a three-year partnership with Oxfam and the Ethiopian Government to design and test a feedback-based performance management system for the Ethiopian extension system to make it more effective and more farmer-driven. Here is a very brief summary of what it might look like:

How data is gathered

There are two main ways of gathering performance data from farmers and other stakeholders.

1. Every one or two years, independent and anonymous surveys are conducted with a representative sample of farmers. The surveys provide a safe space for farmers to give their honest perceptions and ideas for improvement (using a rating scale of 0-10 as well as open written responses) on three aspects of extension performance:

• The relevance and quality of services.
• The quality of their relationship with the extension officers.
• The perceived impact on their livelihoods.

2. On-going micro-surveys are conducted at key touch points (e.g. a training session). Micro surveys are ideally suited to automated data gathering technologies such as SMS and handheld devices which reduces costs and improves reliability and timeliness. They create a continuous stream of real-time performance data on the farmer experience of extension activities, relationship quality and impacts on livelihoods for management decision-making.

In other words, while the comprehensive surveys provide detailed periodic snapshots, the continuous feedback allows you to play the movie from the perspective of primary constituents and other stakeholders against key indicators.

How the data is used

The data from both sources can be fed into a central database where it can be aggregated and analysed using statistical methods. It is then possible to automatically generate comparative performance reports at any level of the system (e.g. the performance of a specific agency, a district office or a provincial department).

Keystone has adapted a really simple but powerfully proven methodology from the customer satisfaction industry called Net Promoter Analysis. NPA classifies respondents into promoters, passives and detractors and calculates a single Net Promoter Score. Data presented in this simple but highly effective way enables managers at all levels to manage to the data – and to discuss the feedback with respondents and so deepen insights, make improvements and strengthen relationships.

Figure 1 below is an example from the Ethiopian pilot showing how farmers rated the services provided by the Farmer Training Centre (FTC) and the extension officers that serves them. The ratings for this FTC are compared with the average rating of all the FTCs in the district. In a properly designed report, a much clearer graph is explained with clear and simple text bullets, and illustrated with statements from the open feedback farmers gave as well as their ideas for improvement.

Figure 2 provides another example that compares how different FTCs were rated by the farmers they each serve (the others can be identified or kept anonymous). This data can be further aggregated to compare farmer-rated performance of provincial departments too. 

It is also possible to gather and compare farmer perceptions of other service providers as well such as NGOs and private companies – and to include questions on, for example, the quality of collaboration. Doing so, and reporting publicly in this way could provide a huge incentive to behave in a more collaborative way.

Completing the feedback loop: dialogue for improvement

The real power of NPA and benchmarking lies in the possibilities it offers for reporting back to constituents and facilitating collective sense-making and dialogue for learning and improving. This fosters the kind of transparency and accountability to constituents that builds confidence and trust and enhances the credibility of the program among constituents. 

When organizations report publicly in this way a new kind of learning conversation becomes possible between constituents and stakeholders. Even poor and illiterate communities are able to understand and make sense of feedback data. Our early experience of this kind of data-based dialogue is that it has led to deepened insights, new mutual commitments, improved program activities and strengthened relationships.

Having hard quantifiable performance data of this kind might make it easier for bureaucratic management to be a little more flexible in encouraging innovations that lead to better ratings rather than pre-defined ‘top down’ targets and performance indicators.

Would anyone be interested in a South African pilot of Constituent Voice? 

The principle of independent feedback does not have to be expensive. In a few of our pilots, Keystone is training and using community volunteers, students, and other local data collectors to protect anonymity and confidentiality of respondents and to make the feedback more reliable.

Links

Downloads