An Agent-Based Methodology with Autonomous Evaluation for Knowledge Extraction from Technical Texts
Retrieval-Augmented Generation, Large Language Models, Multi-Agent Systems, Industrial Network, Knowledge Retrieval, Technical Documentation.
This work develops a methodology for retrieving and synthesizing knowledge from complex technical documentation using generative AI architectures. The approach is investigated by analyzing and comparing three configurations: (i) a Large Language Model (LLM), (ii) a single-agent Retrieval-Augmented Generation (RAG) model, and (iii) a multi-agent RAG configuration. To validate this methodology, a representative case study was applied using technical texts from the PROFIBUS protocol. Performance is evaluated through quantitative linguistic metrics (ROUGE, METEOR, MATTR, BERTScore) and qualitative assessments (LLM-as-a-Judge). The results indicate that multi-agent orchestration improves contextual accuracy and content structure in processing complex industrial texts, demonstrating how generative AI can support knowledge-driven, human-centric automation in line with Industry 5.0 goals.