Banca de QUALIFICAÇÃO: EMELYN CLEMENTINO FREIRE

Uma banca de QUALIFICAÇÃO de MESTRADO foi cadastrada pelo programa.
STUDENT : EMELYN CLEMENTINO FREIRE
DATE: 20/08/2026
TIME: 09:00
LOCAL: https://meet.google.com/kma-bwen-qyh
TITLE:

Semantic Indexing in Retrieval-Augmented Generation (RAG) Systems: An Experimental Evaluation Using Academic Dissertations

KEY WORDS:

Retrieval-Augmented Generation (RAG); Semantic Indexing; Semantic Enrichment; Revised Bloom’s Taxonomy; Experimental Evaluation.

PAGES: 50
BIG AREA: Engenharias
AREA: Engenharia Elétrica
SUMMARY:

Retrieval-Augmented Generation (RAG) systems have emerged as an effective approach for enhancing the ability of Large Language Models (LLMs) to retrieve and use information from external document repositories. However, the quality of the generated responses directly depends on the segmentation, indexing, and retrieval strategies employed, particularly when dealing with lengthy and structurally complex academic documents. This research proposes an experimental evaluation of two indexing architectures for RAG systems applied to a corpus comprising 20 dissertations from the Federal University of Rio Grande do Norte (UFRN). A monolithic approach based on uniform text segmentation will be compared with a modular approach semantically enriched through metadata and structural information. The experiments will employ different embedding models and retrieval configurations, with responses generated locally. The evaluation will be conducted using a set of questions organized according to the cognitive levels of the Revised Bloom’s Taxonomy, enabling a systematic analysis of the systems’ retrieval and generation capabilities. Performance will be measured using metrics from the RAGAS framework, complemented by an LLM-as-a-Judge model and statistical tests, including Analysis of Variance (ANOVA). The study is expected to demonstrate that semantic enrichment strategies improve contextual retrieval and the quality of generated responses, while also establishing a reproducible methodology for the experimental evaluation of RAG systems applied to academic collections.

COMMITTEE MEMBERS:
Presidente - 1837240 - MARCELO AUGUSTO COSTA FERNANDES
Externo à Instituição - GABRIEL BEZERRA MOTTA CÂMARA - UFRN
Externo à Instituição - SERGIO NATAN SILVA - UFCG