Tehničko veleučilište u Zagrebu · Zagreb

Usage of Chatbot Techniques for Automation of Knowledge Tests

stručni rad

stručni rad

Usage of Chatbot Techniques for Automation of Knowledge Tests

Vrsta prilog sa skupa (u zborniku)
Tip stručni rad
Godina 2025
Nadređena publikacija 48th ICT and Electronics Convention MIPRO 2025
Stranice str. 383-388
ISSN 1847-3946
Status objavljeno

Sažetak

AI chatbot is often used for user support. It
helps end-users in specialized tasks, diminishing the need for
human assistance in support centers. Two mechanisms are
used for chatbot implementation: RAG (Retrieval
Augmented Generation) and LLM (Large Language Model)
fine-tuning. RAG mechanism employs a prompt for serving
relevant context, selected from the vector database to the
LLM with the user query. After that, LLM constructs
informative replies and returns them to the end-user. It is
important to note that LLM has not been trained on
significant materials in the area where the chatbot is
constructed. Similar procedures can be used to automate
knowledge tests. When the student answers the question
relevant documents are retrieved from a specialized vector
database with teaching materials and serve as a context with
the student's answer to LLM through the prompt. LLM
decides if the answer is correct based on the context and the
student's answer. The procedures of vector database
creation, test creation, and student answering can be
automated, so that minimal interaction is needed from the
teacher.
In the paper, we explain software architecture and the
implementation of such a system, with some
recommendations. At the end of the paper, we present the
proof-of-concept and directions for future development of the
tool.

Ključne riječi

artificial intelligence (AI) in education; chatbots; large language models (LLMs), retrieval-augmented generation (RAG)