Banca de DEFESA: KAROLAYNE SANTOS DE AZEVEDO

Uma banca de DEFESA de DOUTORADO foi cadastrada pelo programa.
STUDENT : KAROLAYNE SANTOS DE AZEVEDO
DATE: 06/08/2026
TIME: 14:00
LOCAL: Remoto
TITLE:

A Hybrid AI Approach Based on Self-Supervised Embeddings, Vehicle Telemetry, and Ocular Markers for Driver Monitoring in Semi-Autonomous Vehicles

KEY WORDS:

Semi-autonomous vehicles; Artificial intelligence; Gaze density maps; Self-supervised embeddings; DINOv2; Vehicle telemetry; Driving style; Cognitive load; Blink rate; Driver monitoring.

PAGES: 140
BIG AREA: Engenharias
AREA: Engenharia Elétrica
SUMMARY:

The introduction of automated and semi-autonomous driving systems has redefined the driver's role, shifting it toward supervision and selective intervention and imposing new demands on attention, engagement, and situation awareness. This scenario calls for non-invasive methods able to characterize, in an integrated way, the vehicle's operating mode and the driver's driving style, addressing a recurring gap in the literature, which typically treats the visual, vehicular, and physiological dimensions in isolation. This thesis proposes and validates a hybrid multi-technique Artificial Intelligence (AI) approach, organized into three complementary layers and grounded in data from the Low Arousal and Sleepiness (LAS) experiment, conducted in a semi-dynamic driving simulator with 26 participants. The first layer replaces hand-crafted ocular metrics with gaze density maps encoded by self-supervised embeddings extracted from the DINOv2 Vision Transformer and compared against multimodal CLIP representations across temporal windows of 30, 60, 150, and 180 seconds. When fed into supervised classifiers (Random Forest, Support Vector Machine, Decision Tree, and a 1D Convolutional Neural Network) and into unsupervised K-Means analysis, these embeddings discriminate the manual and autonomous driving modes with 93.15% accuracy and an F1-score of 0.9254 (SVM, 150 s window), evidencing that discrimination emerges from sustained temporal dynamics and from inter-individual heterogeneity. The second layer characterizes the driving style from six vehicle telemetry parameters acquired at 125 Hz, namely mean speed, standard deviations of acceleration, accelerator pedal and steering-wheel angle, mean lane offset, and number of lane changes, aggregated into three-minute windows, segmented by K-Means (k = 3), and synthesized into an interpretable scalar metric: the polygon area of the radar chart, complemented by a weighted aggressiveness score. The resulting three regimes (stable, moderate, and aggressive) were statistically validated by Kruskal-Wallis tests with pairwise Mann-Whitney comparisons under Bonferroni correction, yielding significant differences (p < 0.05) in four of the six variables (mean speed, H = 66.12; pedal variability, H = 21.01; acceleration variability, H = 18.00; lane changes, H = 9.36). The third layer articulates these regimes with non-invasive ocular physiological markers, the blink rate per minute and the Percentage of Eye Closure (PERCLOS), revealing a monotonic drop in blink rate across regimes (H = 19.5; p < 0.001) accompanied by a statistically stable PERCLOS, a pattern consistent with the attentional blink inhibition signature under high cognitive load rather than with strict fatigue. Taken together, the results show that the proposed representation coherently and interpretably captures the driver's cognitive state on two complementary scales, namely operating mode and driving style, using minimally invasive signals. The main contribution is the systematic integration of these three layers into a single pipeline, which, to the best of the literature reviewed, has not been previously reported, articulating self-supervised visual representations, vehicle telemetry, and physiological ocular markers. The main limitations concern the dependence on the simulation environment, the tripartite granularity of the classification, and the absence of road context, opening directions for naturalistic validation, soft clustering, and the integration of additional physiological markers.

COMMITTEE MEMBERS:
Presidente - 1837240 - MARCELO AUGUSTO COSTA FERNANDES
Interno - 2885532 - IVANOVITCH MEDEIROS DANTAS DA SILVA
Externo ao Programa - 3083298 - RENAN CIPRIANO MOIOLI - UFRNExterna à Instituição - DEBORA CHRISTINA MUCHALUAT SAADE - UFF
Externa à Instituição - LUCIANA RIBEIRO VELOSO - UFCG