Banca de QUALIFICAÇÃO: GABRIEL AFONSO FREITAS AIRES

Uma banca de QUALIFICAÇÃO de MESTRADO foi cadastrada pelo programa.
STUDENT : GABRIEL AFONSO FREITAS AIRES
DATE: 11/05/2026
TIME: 16:00
LOCAL: Google Meet - https://meet.google.com/urn-abst-shm
TITLE:

Cost–Performance Ratio of LLMs in BioNER: Analysis of Scalability and In-Context Learning Strategies in the BC5CDR Corpus

KEY WORDS:

BioNER; Large Language Models; In-Context Learning; scalability; F1-Score; BC5CDR.

PAGES: 105
BIG AREA: Ciências Exatas e da Terra
AREA: Ciência da Computação
SUBÁREA: Sistemas de Computação
SUMMARY:

This work investigates the cost–performance ratio and cognitive limits of Large Language Models (LLMs) in the task of Biomedical Named Entity Recognition (BioNER) under the In-Context Learning (ICL) paradigm. The consolidated BC5CDR corpus (1,500 articles) was used for the structured extraction of chemical compounds (Chemicals) and disease entities (Diseases). Through a distributed and reproducible pipeline based on the vLLM engine, 18 open-weights architectures (1B to 70B parameters) were evaluated under systematic variation of the number of few-shot examples in context (k ∈ {0, 1, 2, 4, 8, 16, 32}). Evaluation was based on the strict Exact Match criterion, measuring Precision, Recall, and F1-Score. The quantitative results reveal that: (i) parametric scalability improves performance, with the 70B model reaching the predictive ceiling (F1 ∼ 0.63), whereas highly instructed 8B models (F1 ∼ 0.61) establish the optimal Pareto frontier, indicating that training refinement compensates for inference cost; (ii) adding few-shot examples induces attentional saturation in smaller architectures, causing collapses of up to 75% in F1-Score under extreme densities (k = 32), a phenomenon quantified by the Context Stability metric (∆); and (iii) there is asymmetry between classes, with Chemicals extracted with high morphological precision (F1 ∼ 0.79), while Diseases (F1 ∼ 0.48) constitute the main barrier of semantic abstraction. The study thus offers guidelines for the efficient and scalable deployment of LLMs in biomedical applications.

COMMITTEE MEMBERS:
Presidente - 2353000 - ELIAS JACOB DE MENEZES NETO
Interno - 2668551 - ANDRE MORAIS GURGEL
Interno - 2180207 - ITAMIR DE MORAIS BARROCA FILHO