Evaluating social and development interventions using the Qualitative Impact Protocol (QuIP)
A reality check of extent to which self-reported changes in wellbeing can be attributed to specific social and development interventions in complex contexts.
Understanding the effectiveness of social and development interventions
The Qualitative Impact Protocol (QuIP) has been designed to offer a credible and cost-effective assessment into the extent that changes in wellbeing that have been self-reported by intended beneficiaries can be attributed to specified social and development interventions in complex contexts. Action research into its effectiveness is being undertaken through commissioned impact evaluation studies undertaken by the Centre for Development Studies, often through a spin-out social enterprise called Bath Social and Development Research.
Evaluating project impact
International development agencies need credible and cost-effective evidence of their socio-economic impact to learn how to improve on what they do, and also to account for their activities to the public. QuiP addresses this need for impact evaluation, particularly in complex contexts, both as a substitute and complement to randomised control trials and other methods.
The QuIP was an output of the collaborative Department for International Development (DFID) ESRC sponsored ART Project (Assessing Rural Transformations). This ran from 2012-2015 and entailed collaboration with three international NGOs and researchers from three African Universities.
After 2015 the QuIP was used to engage with international development agencies and evaluators over impact evaluation methodology, both challenging poor practice and offering a feasible alternative. Up to the end of 2018, nearly 40 conference, seminar or training presentations based on the QuIP had been delivered in more than a dozen countries.
Bath Social and Development Research Ltd (BSDR) was set up in 2016 as a non-profit spin-off company of the University to promote wider. By the end of 2018 it had conducted or collaborated in more than 25 QuIP studies across 15 countries.
Enabling change
The main impact of the research arises from induced changes in the activities of organisations that commission QuIP studies, and the effect of these on the wellbeing of intended beneficiaries of their subsequent activities. It also includes savings arising from the greater cost-effectiveness of QuIP relative to other methods, and the intrinsic benefit of providing intended beneficiaries with a credible feedback mechanism for expressing their views about what development agencies are doing in their name.
Other impacts are conceptual, instrumental and entail capacity building.
Conceptual
The QuIP research has provided a detailed, visible and practical example of how credible and cost-effective evidence can be generated on the impact of development motivated action. It has strengthened understanding of what can be achieved through qualitative, mixed method and realist approaches to impact evaluation compared to both more expensive and inflexible experimental approaches.
It has also challenged qualitative evaluation specialists to be more systematic and transparent in their methods. It has contributed to debate over the ethics and efficacy of blinded qualitative research, and attribution coding.
Instrumental
The QuIP has been directly adopted by a range of development agencies, including international non-government development organisations, social investors, philanthropic foundations, a for-profit company and local authorities in the UK. It has had a wider influence on the practice of impact evaluation – by other organisations, including private consultancies (Oxford Policy Management), foundations (the Aga Khan Development Network) and official donors (DFID).
Capacity building
The research has led to the establishment of a non-profit company (BSDR) and through it a network of professional evaluators, University based qualitative researchers and analysts – principally across Africa, but also in India and Peru.