The widespread use of large language models (LLMs) in education has introduced challenges related to citation reliability, academic integrity, and factual accuracy. This study presents the Augmented LLM-Based Engagement System (ALES), a domain-specific chatbot designed for higher education. ALES incorporates retrieval-augmented generation (a method for grounding responses in verified documents), citation-aware output, and ethical usage safeguards. The system's design was informed by a structured state-of-the-art analysis of 584 articles, of which 76 peer-reviewed studies were selected for analysis based on strict inclusion criteria. The review identified critical limitations in current academic AI systems, including lack of transparency, source verifiability, and institutional integration. ALES addresses these gaps through a modular architecture and university-compatible interface. Preliminary comparison with existing academic AI tools highlights ALES's potential to support student learning while promoting responsible use. This paper outlines the platform's architecture and proposes directions for future development and evaluation.

The ALES Platform: State of the Art and Gap Analysis for an Academic LLM Chatbot

Kartikee Awasare
Conceptualization
;
Antonella Petrillo
Supervision
;
Mizna Rehman
Writing – Original Draft Preparation
2025-01-01

Abstract

The widespread use of large language models (LLMs) in education has introduced challenges related to citation reliability, academic integrity, and factual accuracy. This study presents the Augmented LLM-Based Engagement System (ALES), a domain-specific chatbot designed for higher education. ALES incorporates retrieval-augmented generation (a method for grounding responses in verified documents), citation-aware output, and ethical usage safeguards. The system's design was informed by a structured state-of-the-art analysis of 584 articles, of which 76 peer-reviewed studies were selected for analysis based on strict inclusion criteria. The review identified critical limitations in current academic AI systems, including lack of transparency, source verifiability, and institutional integration. ALES addresses these gaps through a modular architecture and university-compatible interface. Preliminary comparison with existing academic AI tools highlights ALES's potential to support student learning while promoting responsible use. This paper outlines the platform's architecture and proposes directions for future development and evaluation.
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11367/153463
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact