How do we support farmers to replant after a flood? Give a platform for unemployed young people to find suitable work? Help tenants to overcome housing problems? Or assist women to escape domestic violence?

Finding the best ways to help people facing problems resulting from social injustice, economic shocks and crises - like COVID-19 and climate change - is not easy, whoever and wherever they are. While there's no easy answer, an essential factor in effective action is listening carefully to those being offered help. Satellite pictures and online surveys can tell us some things, but they are no substitute for hearing how those on the ground themselves describe what is happening to them and why.

For more than ten years, Professor James Copestake at the Centre for Development Studies has collaborated with development organisations around the world to find reliable and cost-effective ways to do precisely this.

Reducing bias with the QuIP

The result is the Qualitative Impact Protocol (QuIP). This involves gathering qualitative data (using interviews and focus group discussions) and analysing it in transparent ways to identify how respondents experience the effects of development assistance on their lives and livelihoods. Field staff gather the information with the minimum of reference to the project or programme being evaluated, helping to reduce confirmation bias and to generate more balanced findings.

Action research with the QuIP in Malawi and Ethiopia led to the formation of a non-profit company – Bath Social and Development Research Ltd. (Bath SDR) - to encourage its use more widely and sustainably. Five years later the QuiP has been used to bring local voices to the fore across more than 20 countries, including the UK.

‘It paints an honest picture of what's going on from the perspective of our intended beneficiaries.’ - Everett Peachey, Acting Director of the Aga Khan Development Network’s Quality of Life Unit

How the QuIP works

QuIP interviews and discussions start by finding out what changes (positive and/or negative) in their lives participants choose to highlight. Experienced field staff then collect and record people’s own explanations for why these things have happened.

The field staff usually do not belong to the organisation seeking the feedback, and often don't even know the organisation. This independence helps to secure fuller and more reliable evidence of what is happening.

The stories obtained are carefully translated, coded, analysed and turned into graphics that summarise what is happening. Bath SDR has been at the forefront of designing and testing a software package called Causal Map to do this.

The organisation sponsoring the QuIP then compares these findings with what it previously thought. This helps them to act on differences to improve their work.

'One of the reasons we chose the QuIP was the fact that the adolescent girls themselves would have a chance to share their own stories of change, hence making their perspectives heard. QuIP also worked better for this study due to our constrained budget.' - Rosemary Nyaga Monitoring and Evaluation Manager, Feed the Children

Why the QuIP is needed

Development organisations can easily fall into the trap of simplifying what is really going on in the areas where they work. This can lead to assumptions that their own actions are more important than they really are, for example. Left untested, such biases can lead to inefficient or even irrelevant efforts. QuIP studies serve as a reality check on whether the ideas behind their plans are really fit for purpose.

The QuIP is just one of many ways of generating such feedback. It is different to others because it focuses on finding out why people believe things have happened to them, rather than just looking at what has happened. By doing this, it often reveals unintended consequences and otherwise unseen causes. It is also unusual in being the product of systematic and sustained action research.

‘Resilience and adaptation - what do these words mean to the people actually affected? I think QuIP gives community participants a voice on these in a way that we can use to improve programmes.’ - Martin Whiteside, Independent Consultant

When it comes to concepts that are hard to define - such as resilience, sustainability and empowerment - the QuIP is one of the few methods that offers a way to identify how they are personally experienced, and what they mean in the words of those coping with crises, shocks and injustices. Do they feel more ‘empowered’ or ‘resilient' and why?

`The QuIP’s approach ensures organisations, and the wider international development community, hear the stories of the people they aim to help.

What next for the QuIP

In 2019 the Centre for Development Studies, Bath SDR and the publisher Practical Action produced ‘Attributing Development Impact’. This is a free e-book that shares experiences of using the QuIP in the UK, Ethiopia, India, Malawi, Mexico, Tanzania and Uganda. The book also provides detailed guidelines for conducting a QuIP and reflects on the methodological challenges it addresses.

Bath SDR continues to train and support organisations and professional evaluators in how to conduct QuIP studies. One trainee was Rosemary Nyaga from Kenya, who then used the QuIP to conduct a study of the Kajiado Nutrition Campaign, a programme addressing intergenerational cycles of malnutrition, undernourishment, obesity, early marriage and childbearing.