Banca de QUALIFICAÇÃO: RAMON AUGUSTO SOUSA LINS

Uma banca de QUALIFICAÇÃO de DOUTORADO foi cadastrada pelo programa.
DISCENTE : RAMON AUGUSTO SOUSA LINS
DATA : 12/12/2017
HORA: 09:00
LOCAL: Núcleo de Pesquisa e Inovação em Tecnologia da Informação - NPITI
TÍTULO:

Q-learning with Deep Learning Approximator Applied to the k-Servers Problem

PALAVRAS-CHAVES:

K-Server Problem, Reinforcement Learning, Q-Learning Algorithm, Deep Learning

PÁGINAS: 67
GRANDE ÁREA: Engenharias
ÁREA: Engenharia Elétrica
RESUMO:

The k-server problem is perhaps the most influential problem of online computing whose solution remains open. Its conceptual simplicity contrasts with its complexity that grows fast with the increasing numbers of nodes and servers. In an attempt to overcome this problem, we propose to use the Q-learning algorithm in conjunction with deep convolutional neural network. Preceded to this work, the Q-learning was used in the solution of small instances and compared to main algorithms (Work function and Harmonic) used in solving online problems. For large instances, Q-learning was used in a hierarchical way, where a number of nodes and servers (constant) were separated into clusters. The local policy obtained in each cluster was used in the formation of a global policy, being able to approach the problem for greater instances. Both methods fall into the curse of dimensionality problem and a new problem arises in the hierarchical approach when requisitions happen in distinct clusters, generating states not visited during training. This problem was solved using the greedy strategy, where the displaced server is the least distant of the request. Our objective is to use the deep reinforcement learning concept, more specifically Q-learning with deep convolutional neural network, to calculate the state-action value function Q in terms of the synaptic weights of the network. This approach eliminates the need to calculate the Q table that refers to all possibilities. With that, it is hoped that the curse of dimensionality would be diminished by rendering dimensionalities until now intractable. Another objective is to use this generalization capacity instead of the greedy method for hierarchical solution, in order to infer better server displacements in
distinct clusters.

MEMBROS DA BANCA:
Presidente - 347628 - ADRIAO DUARTE DORIA NETO
Interno - 1837240 - MARCELO AUGUSTO COSTA FERNANDES
Interno - 1673543 - SAMUEL XAVIER DE SOUZA
Externo ao Programa - 350241 - JORGE DANTAS DE MELO
Externo à Instituição - FRANCISCO CHAGAS DE LIMA JUNIOR - UERN