Budget
£469,256
Project status
In progress
Duration
1 Sep 2026 to 31 Aug 2027
£469,256
In progress
1 Sep 2026 to 31 Aug 2027
Research assessment systems such as the REF make high-stakes funding decisions by evaluating individual research outputs. But considering a single study in isolation can be misleading: statistically significant findings can arise by chance, through flexible analytical choices, or because only the most "successful" results get written up.
This project starts from the idea that the health of the wider system producing research outputs provides essential context for judging how far any individual claim can be trusted. In a healthy research environment, impressive results are harder to generate by accident.
REF assessment panels currently have few, if any, tools to help make that distinction.
We propose Rigour Hygiene Indicators: population-level, interpretable measures of the "evidential environment" surrounding a body of research. Rather than judging individual papers or researchers, they diagnose the statistical plausibility of the environment, such as a REF unit of assessment, from which excellent work emerges, providing missing context to support, not replace, expert judgement.
Develop and validate a core Rigour Hygiene Indicator based on the logic of excess statistical significance, calibrated through simulation and tested against REF 2021 outputs in Psychology, Psychiatry and Neuroscience.
Build an automated pipeline that assembles, retrieves and quality-checks statistical reporting features from thousands of research outputs, combining established extraction tools with LLM-assisted annotation.
Extend the framework with a prototype indicator for sex/gender-disaggregation practices in clinical research, and publish an assessment-ready reporting pack and policy brief for funders, REF panels and institutions.
| Team member | Role | Organisation |
|---|---|---|
| Katherine Button | Project Lead | University of Bath |
| Gordon Fletcher | Project Co-Lead (UK) | University of Salford |
| Xiarui Jiang | Project Co-Lead (UK) | University of Sheffield |
| Pavel Osveiko | Project Co-Lead (UK) | University of Oxford |
| Rossella Salandra | Project Co-Lead (UK) | University of Bath School of Management |
If you have any questions about this project, please get in touch.