HYBRID-RAG VIRTUAL TEACHING ASSISTANT FOR ENGINEERING EDUCATION

Recent advances in large language models (LLMs) have enabled conversational agents that support learning beyond the classroom by answering student questions fluently. However, general-purpose LLMs are prone to hallucinations and non-factual responses when queried about course-specific technical content, and course-by-course fine-tuning is often impractical due to data, compute, and maintenance costs. To address these constraints, we design, deploy, and evaluate a course-scoped Hybrid Retrieval Augmented Generation (HRAG) teaching assistant for a graduate Introduction to Computational Natural Language Processing course at McMaster University. Our hybrid retrieval method combines sparse lexical matching (BM25) with dense embedding based semantic retrieval, fuses the two signals with a weighted scoring rule. This hybrid retrieval pipeline increased the system’s answer accuracy by 17 percentage points, from 65% under a vanilla RAG baseline to 82% with HRAG. In a Winter 2024 deployment, students who used the system achieved higher mean weighted course grades than students without exposure.