Banca de QUALIFICAÇÃO: GABRIEL LUCAS DE MEDEIROS LEITE

Uma banca de QUALIFICAÇÃO de MESTRADO foi cadastrada pelo programa.
STUDENT : GABRIEL LUCAS DE MEDEIROS LEITE
DATE: 05/08/2026
TIME: 10:00
LOCAL: Remoto (Google Meet)
TITLE:

Political Bias in Large Language Models (LLMs): A Critical Review and A Responsible Methodology

KEY WORDS:

political bias; large language models; PRISM; machine learning; TF-IDF.

PAGES: 55
BIG AREA: Ciências Exatas e da Terra
AREA: Ciência da Computação
SUBÁREA: Metodologia e Técnicas da Computação
SPECIALTY: Processamento Gráfico (Graphics)
SUMMARY:

Auditing political bias in Large Language Models (LLMs) has evolved from rigid forced-choice surveys to sophisticated semantic evaluations of open-ended textual responses. Within the PRISM methodology, an LLM judge scores argumentative essays generated on Political Compass Test propositions—a step that is effective but computationally costly when applied at scale. This qualification work investigates whether lightweight supervised classifiers can replicate those judge-assigned silver-standard labels after an initial labeling bootstrap, thereby amortizing inference cost at the PRISM scoring step. Using PRISM-style corpora of LLM-generated political essays (Stage 1: 3,703 essays, 3-class social-axis labels; Stage 2: 22,979 essays, 5-class multilingual labels), we evaluate base classifiers and ensemble methods under several feature-reduction strategies. Reported accuracy measures agreement with the reference LLM judge, not independent human ground truth. A Multilayer Perceptron (MLP) with  χ² feature selection reaches 98.41% agreement on Stage 1 and remains competitive under the harder Stage 2 setting (≈ 87% with ensembles). The results support judge distillation as a viable engineering optimization within PRISM-like pipelines: it amortizes scoring cost but replicates the judge's behaviors and the PCT's priors rather than correcting them. The approach does not eliminate the need for LLM-as-a-Judge during initial label creation or generalize to arbitrary open-ended evaluation tasks without labeled data, but demonstrates the potential for lightweight classification as a preliminary step in PRISM-like pipelines and highlights the possibility of reducing the computational cost of political bias auditing.

COMMITTEE MEMBERS:
Interna - 1350250 - ANNE MAGALY DE PAULA CANUTO
Externo à Instituição - JONATHAN CARDOSO SILVA
Externo à Instituição - JOSÉ GOMES LOPES FILHO
Presidente - 2859562 - LEONARDO CESAR TEONACIO BEZERRA