Banca de DEFESA: CARLOS EDUARDO MORAIS VIEIRA

Uma banca de DEFESA de MESTRADO foi cadastrada pelo programa.
STUDENT : CARLOS EDUARDO MORAIS VIEIRA
DATE: 29/06/2021
TIME: 08:30
LOCAL: https://meet.google.com/hdo-edws-rfz
TITLE:

Assessing irace for automated machine and deep learning in computer vision

KEY WORDS:

Automated machine learning; algorithm configuration; computer vision; deep learning; neural architectural search

PAGES: 75
BIG AREA: Ciências Exatas e da Terra
AREA: Ciência da Computação
SUBÁREA: Teoria da Computação
SPECIALTY: Análise de Algoritmos e Complexidade de Computação
SUMMARY:

Automated machine learning (AutoML) is a field of great interest to both industry and academia. AutoML has allowed developers working on machine learning (ML) applications to achieve satisfactory results with little to no ML expertise. More recently, AutoML tools focused on deep learning (DL) models have proven especially useful to applications where domain-specific algorithms are predominant, as in computer vision (CV) tasks. Still, AutoML tools focused on simpler ML pipelines remain an important alternative, since DL models have high computational resource requirements and offer reduced model interpretability.
Among the main AutoML approaches, AutoML based on an algorithm configurator (AC) is commonly used to produce simpler pipelines, whereas neural architecture search (NAS) is used to produce deep learning models. These two approaches also intersect, since an AC can be used as a NAS algorithm.

In this work, we study the application of the irace AC to both these AutoML methods. The irace configurator has been successfully applied to design effective algorithms for optimization problems, but it has not yet been applied to AutoML. Our assessment is performed in two stages. First, we propose an irace-based AutoML tool to produce simple and effective ML pipelines. The tool is dubbed iSklearn, for which we define a configuration space and setup. We demonstrate that iSklearn is able to produce effective ML pipelines using irace as its AC, with comparable performance to more complex ensembles produced by AutoSklearn, an established configuration-based AutoML tool. Moreover, we show the benefits of the configuration space and setup proposed for iSklearn, even when coupled with another AC. In the second part of our work, we assess irace as a NAS algorithm. To do so, we evaluate irace on NAS-Bench-101, a recent NAS benchmark for the CIFAR-10 CV dataset. We benchmark irace not only as to final-quality assessment, but also as to anytime performance through a bi-objective formulation. Results demonstrate that  irace can be used a NAS algorithm, obtaining comparable results to the best NAS algorithms included in NAS-Bench-101 in terms of final quality. However, further work is required to improve its anytime performance in this context. Finally, we discuss other design choices made in the original benchmark, showing how they affect the relative performance of NAS algorithms, and provide guidelines for improving the assessment of NAS algorithms through the use of NAS-Bench-101.

BANKING MEMBERS:
Presidente - 2859562 - LEONARDO CESAR TEONACIO BEZERRA
Interno - 1669545 - DANIEL SABINO AMORIM DE ARAUJO
Externa à Instituição - LESLIE PÉREZ CÁCERES - PUC
Externo à Instituição - JOAQUIN VANSCHOREN