This guidance provides researchers and staff with practical information on the responsible use of Artificial Intelligence (AI) within research activities. It is intended to support compliance with existing University policies on Research integrity and should be read alongside the UKRIO guide, Embracing AI with integrity. This guide applies to anyone who is involved in research at the University.
Throughout the research lifecycle Artificial Intelligence (AI) tools are becoming increasingly common. They can support activities throughout, however if these tools are used, researchers are expected to have appropriate oversight and judgement regarding their output. Human oversight in any AI usage is essential to support the creation of high-quality research.
*PLEASE NOTE: This guidance is a living document and will be reviewed and updated as AI technologies, sector expectations and good practice evolve. Additional information and examples will be added over time. *
Accountability
All researchers have accountability for their work. As stated in our Authorship and acknowledgement in research outputs it is unacceptable to list any AI or LLMs (Large Language Models), such as ChatGPT, as an author or co-author. All authors are expected to take full responsibility for their work, including any errors that a LLM might have created. Any AI outputs must be reviewed, validated and verified before use.
Data Protection, Confidentiality and Information Security
Researchers must carefully consider the data they input when using any AI tool. Research data may contain personal data, confidential information, commercially sensitive material, or unpublished research findings. Depending on the tool used, any information inputted may be retained or used to improve future versions of the model. This may include the use of the information entered into the tool in future outputs creating a risk that material entered by one user could be reproduced to another. Researchers must not enter sensitive information into any system where it could be disclosed to other users. Researchers remain responsible for information entrusted to them.
Where AI tools such as LLMs are used for your research to process data, it’s important to put in place appropriate safeguards. When Microsoft CoPilot displays the green shield with a tick in the centre, typically in the top right of the chat box, prompts and responses are covered by enterprise data protection and are not used to train the underlying model. For University business, this is the preferred option when using a generative AI tools.
Using AI to generate data and content
The generation of text can be useful to polish text and help those whose first language is not English. AI can also help with planning and brainstorming, as well as structuring your written work. However, text generated from scratch by AI systems may contain hallucinations. Hallucinations are outputs that appear plausible but are factually incorrect. Not only may hallucinations appear if you create text for your research project, if you generate or even reorganise your reference list using AI, hallucinations can also appear in the bibliography. Without accurate citations, your work may not be verifiable and could undermine the credibility of your research. As always, researchers should thoroughly review and verify any AI generated content. AI can be used to create schematics and diagrams. This is appropriate provided the outputs are reviewed and validated. Generating images or other content and presenting them as evidence of real experimentation or research findings constitutes falsification and is not permitted. AI tools used for the assistance of data analysis or interpretation must have appropriate oversight. Researchers remain responsible for the accuracy, validity and reproducibility of any findings generated using these tools – all outputs should be critically evaluated. Researchers must also be aware that AI systems may reflect biases that are present in their training data. All outputs should be critically assessed for these biases before proceeding.
Copyright
The copyright status of AI-generated content remains an evolving area of law. AI-generated images and other content may be included in research outputs, provided their use is transparent, appropriately validated and permitted by the relevant publisher or funder.
Disclosing the use of AI
The use of AI tools at any point during the research lifecycle should be disclosed. In a research paper, disclosure can usually be made in the Methods or Acknowledgements sections. In other formats, for example grants or ethics committee submissions, researchers should include a disclosure wherever relevant. Researchers should retain records of AI use. Transcripts of interactions with AI chatbots should be retained where relevant, together with other relevant AI outputs, and previous versions of the work for audit purposes. This is something that journals can ask for in order to help assess the validity of a claim about how the researcher has used AI. That is for example, if the researcher has a hallucinated reference in their publication, if the journal sees that AI has been used to reformat the references and the LLM has simply regurgitated it wrong, this will mitigate retraction.
Best practice is to include a statement on the use of AI even if confirming that no AI tools were used during the work. It is suggested that, where AI tools were used that the statement includes details of the purpose of its use (e.g. image editing, translation, data analysis, coding assistance, etc), the type of AI tool used (e.g. Microsoft Co Pilot, Claude, ChatGPT, etc), the version/model, date(s) used, how AI contributed and how the outputs were reviewed/validated.
Use of AI tools and human participation in research projects
When AI tools will be used to process, analyse or interpret participant data, researchers should be transparent about this during the consent process. Participants should be provided with sufficient information to understand how AI will be used, what data will be processed, any associated risks, and the measures in place to protect their privacy and confidentiality.
Where appropriate, participant-facing materials may include statements such as:
This project will use Artificial Intelligence (AI) tools to support [describe purpose, e.g. data analysis, transcription, coding or interpretation] . The AI tool(s) will have access to [describe data] . Data will be processed in accordance with applicable data protection requirements and appropriate safeguards will be in place to protect the privacy, confidentiality and security of your information.
There are potential risks associated with the use of AI tools, including [describe relevant risks] . Your data may be processed or temporarily stored within the AI system as part of this research. Further information about these risks and the safeguards in place is provided below (...)
Record keeping
Researchers should consider retaining:
- Copies of prompts used where these materially influenced the research output
- Transcripts of interactions with AI tools where they informed analysis, interpretation, drafting or decision-making
- AI-generated outputs that were incorporated into research outputs, applications or submissions
- Earlier drafts showing how AI-generated content was reviewed, edited and validated by the researcher
- Records of any verification or fact-checking undertaken in relation to AI-generated content
- Details of the AI tool used, including the version or model where known
- Any disclosures made regarding AI use in publications, funding applications or ethics submissions