Banca de DEFESA: GABRIEL CALDAS BARROS E SÁ

Uma banca de DEFESA de MESTRADO foi cadastrada pelo programa.
STUDENT : GABRIEL CALDAS BARROS E SÁ
DATE: 07/11/2024
TIME: 10:30
LOCAL: https://meet.google.com/asd-jgiu-gyu
TITLE:

Jaguar: A Hierarchical Deep Reinforcement Learning approach with Transfer Learning for StarCraft II

KEY WORDS:

Deep Reinforcement Learning; StarCraft II; Multi-Agent; Transfer Learning; Hierarchical Architecture.

PAGES: 90
BIG AREA: Ciências Exatas e da Terra
AREA: Ciência da Computação
SUMMARY:

Real-Time Strategy Games are environments that generally simulate real military situations and present a series of challenges for the field of Artificial Intelligence, such as high complexity, large action and state spaces, partially observable maps, dealing with multiple units at the same time, and also the fact that tasks can be performed within the scope of micromanagement or macromanagement. In particular, Reinforcement Learning has stood out in the application and evolution of techniques capable of dealing with these challenges. A systematic review of the literature was then carried out to understand the state-of-the-art use of Deep Reinforcement Learning in Real-Time Strategy games. Some of the most relevant information raised were the use of StarCraft II as the main simulation environment, the need for more studies addressing macromanagement, the good performance of hierarchical architectures, and the importance of using action masking and transfer learning techniques, which can reduce the complexity of the problem and the computational cost. Given this and the challenges in this field, this work proposes Jaguar, a hierarchical architecture capable of handling both macro and micromanagement, while requiring few resources during training. Jaguar implements techniques for action and state shaping, invalid action masking through the state and the communication between hierarchical levels, different types of rewards, and uses the DQN model for micro decisions and DDQN for strategic decisions. The agent was trained in a base scenario and transfer learning was applied to two other scenarios. The results show that the agent was able to perform both micromanagement and macromanagement, presenting good learning in the base scenario and promising results in subsequent scenarios, demonstrating that the proposed approach was able to deal significantly well with the complexity of StarCraft II scenarios using few computational resources, leaving room for future improvement.

COMMITTEE MEMBERS:
Presidente - 2978747 - CHARLES ANDRYE GALVAO MADEIRA
Interno - 1669545 - DANIEL SABINO AMORIM DE ARAUJO
Externo ao Programa - 1837240 - MARCELO AUGUSTO COSTA FERNANDES - UFRNExterno à Instituição - VINCENT CORRUBLE - SU