Banca de QUALIFICAÇÃO: ANGELO MARCELINO CORDEIRO

Uma banca de QUALIFICAÇÃO de MESTRADO foi cadastrada pelo programa.
STUDENT : ANGELO MARCELINO CORDEIRO
DATE: 14/09/2026
TIME: 16:00
LOCAL: Google meet
TITLE:

Fault Tolerance in High-Performance Computing for Distributed Large Language Model Training

KEY WORDS:

fault tolerance; checkpointing; heartbeat; distributed training; large language models; data parallelism; high-performance computing; preemptible instances.

PAGES: 45
BIG AREA: Ciências Exatas e da Terra
AREA: Ciência da Computação
SUMMARY:

The distributed training of large language models (LLMs) in High-Performance Computing (HPC) architectures faces significant cost and fault tolerance challenges. As the number of compute nodes increases, so does the probability of failures, while cloud instance preemption and hardware faults can interrupt entire training runs and result in substantial computation loss. This research proposes the integration of DeLIA (Dependability Library for Iterative Applications), an application-level fault tolerance library for HPC, into data-parallel LLM training through the DeLIApy Python wrapper. The methodology comprises four stages: DeLIApy packaging and distribution, baseline training setup, integration and overhead evaluation, and resilient execution in the presence of failures. As a contribution, this work provides a fault tolerance approach accessible to the machine learning ecosystem and empirically evaluates its applicability to distributed LLM training.

COMMITTEE MEMBERS:
Interna - ***.034.615-** - CARLA DOS SANTOS SANTANA - UFRN
Interno - 2885532 - IVANOVITCH MEDEIROS DANTAS DA SILVA
Interno - 1673543 - SAMUEL XAVIER DE SOUZA
Externo à Instituição - ITALO AUGUSTO SOUZA DE ASSIS - UFERSA