Retrieval-Augmented Generation Enhanced Smart Training Chatbot: Integrating Elasticsearch for Better Retrieval

Integrating Elasticsearch as the retriever in a Retrieval-Augmented Generation (RAG) framework is essential for improving chatbot responses' factual accuracy and contextual relevance. Traditional Large Language Models (LLMs) often fall short of providing up-to-date information, especially in dynamic domains like training. To meet the demands of smart training, Elasticsearch is integrated into the RAG framework as the retrieval module to enable efficient access to up-to-date and context-specific information. System-level experiments and a user-centered survey have been conducted to evaluate the effectiveness of system. Technical performance is assessed through comparisons with traditional retrieval methods, while user acceptance is evaluated by using the Technology Acceptance Model (TAM). System-level experiments show that integrating Elasticsearch into the RAG framework substantially advances chatbot performance, with the F1-score increasing from 0.47 to 0.85 and response time decreasing from 1.5 to 0.8 seconds. User satisfaction improves from 31% to 82%. The user-centered survey based on the TAM confirms the system's effectiveness, with all four constructs demonstrating acceptable reliability (Cronbach's α > 0.65) and composite reliability (> 0.79). Multiple regression analysis reveals that Attitude Toward Use (ATT; β = 0.280, p <.001) and Perceived Usefulness (PU; β = 0.279, p <.001) significantly predict users' Behavioral Intention (BI) to use the system. These findings indicate that integrating Elasticsearch into the RAG framework advances retrieval accuracy and efficiency. Furthermore, it enhances user acceptance in bilingual smart training chatbot applications, improves the learning experience, and offers a practical approach to advancing smart training.