Integration of Artificial Intelligence (AI) into the Data Extraction Phase of a Scoping Review

This paper describes how artificial intelligence (AI) was used to assist with the data extraction phase of a scoping review, specifically comparing different AI models and the accuracy of AI-assisted data extraction compared to human extraction. Scoping reviews map existing literature on a topic and are useful for complex or under-reviewed subjects. Integrating AI, particularly large language models, can enhance processing speed and data analysis. Three models, ChatGPT 3.5 and -4 (both developed by OpenAI) and Copilot (by Microsoft), were compared to identify the best model for AI-assisted data extraction. Adobe Acrobat Pro’s Optical Character Recognition (OCR) feature and ‘ChatGPT Splitter’ were used to manage image-based content and large sections of data. A custom script was iteratively generated and implemented with the source material. AI-assisted extraction results were compared to text extracted by an independent reviewer. ChatGPT-4 was utilized to enhance efficiency and accuracy of data extraction from 234 sources. While human extraction was more specific with verbatim information, AI was faster and sometimes provided more nuanced understanding, averaging 20 minutes per source compared to one hour for human extraction. ChatGPT-4’s superior text processing capabilities made it the optimal choice. While AI advancements have streamlined data extraction, human oversight remains crucial to ensure accuracy and address biases. This methodology is especially beneficial for smaller research teams and emphasizes the importance of structured prompts and rigorous review. Careful planning and oversight can mitigate risks, ultimately improving the quality and efficiency of the review process.