SEIT-Agent: An experience memory enhanced inspection and testing agent for high-safety special equipment with risk-aware LLM evaluation

The inspection and testing of high-safety special equipment (HSSE) is crucial for ensuring its safe and stable operation. However, two core tasks still require substantial manual effort: damage mode analysis requires synthesizing dispersed damage-related knowledge, while equipment quality grading involves determining and applying standard-specified rules. Existing LLM-based research augmented with RAG mainly focuses on information retrieval and Q&A, lacking mechanisms to autonomously and deterministically execute context-dependent rules. To address these challenges, this paper proposes the special equipment inspection and testing agent (SEIT-Agent), an AI Agent framework that integrates an LLM-based planner, a specialized tool library, an experience memory pool and a domain knowledge base. The tool library encapsulates quality grading rules for reliable autonomous grading, while the knowledge base integrates damage mode information and is accessible to the LLM via a hybrid vector–keyword RAG approach. The experience memory pool stores successful task execution trajectories to enhance consistency and accuracy in repetitive quality grading tasks. Furthermore, we introduce an expert-guided risk-aware LLM evaluation strategy where domain experts define critical and supporting assessment points and an external large-scale LLM evaluates response quality under a critical-point gating mechanism. Experiments on real-world data from an HSSE enterprise demonstrate that SEIT-Agent outperforms baseline methods on two core tasks.