Skip to main content

Rigour Hygiene Indicators for Research Assessment

Building population-level diagnostics of rigour to give the UK's Research Excellence Framework panels, funders and institutions context for judging excellence.

Budget

£469,256

Project status

In progress

Duration

1 Sep 2026 to 31 Aug 2027

Project overview

Research assessment systems such as the REF make high-stakes funding decisions by evaluating individual research outputs. But considering a single study in isolation can be misleading: statistically significant findings can arise by chance, through flexible analytical choices, or because only the most "successful" results get written up.

This project starts from the idea that the health of the wider system producing research outputs provides essential context for judging how far any individual claim can be trusted. In a healthy research environment, impressive results are harder to generate by accident.

REF assessment panels currently have few, if any, tools to help make that distinction.

We propose Rigour Hygiene Indicators: population-level, interpretable measures of the "evidential environment" surrounding a body of research. Rather than judging individual papers or researchers, they diagnose the statistical plausibility of the environment, such as a REF unit of assessment, from which excellent work emerges, providing missing context to support, not replace, expert judgement.

What we'll do

Develop and validate a core Rigour Hygiene Indicator based on the logic of excess statistical significance, calibrated through simulation and tested against REF 2021 outputs in Psychology, Psychiatry and Neuroscience.

Build an automated pipeline that assembles, retrieves and quality-checks statistical reporting features from thousands of research outputs, combining established extraction tools with LLM-assisted annotation.

Extend the framework with a prototype indicator for sex/gender-disaggregation practices in clinical research, and publish an assessment-ready reporting pack and policy brief for funders, REF panels and institutions.

Team members

Team member Role Organisation
Katherine Button Project Lead University of Bath
Gordon Fletcher Project Co-Lead (UK) University of Salford
Xiarui Jiang Project Co-Lead (UK) University of Sheffield
Pavel Osveiko Project Co-Lead (UK) University of Oxford
Rossella Salandra Project Co-Lead (UK) University of Bath School of Management

Funders

Contact us

If you have any questions about this project, please get in touch.