CONVOLUTIONAL NEURAL NETWORK APPLIED TO RECOGNIZING LIBRAS HAND CONFIGURATIONS FROM THE FREQUENCY RESPONSE OF AN FSS
FSS; Machine Learning; Convolutional Neural Network; Brazilian Sign Language (Libraas), Sign Language.
This thesis addresses the implementation, training, and testing of a 2D neural network with 3 convolutional layers and 3x3 dimension filters employed to identify American Sign Language (ASL) hand configurations. The data feeding the neural network consists of images of the transmission coefficient (S11) graph of an FSS Data Glove created using denim fabric and conductive ink. During the testing of the FSS Data Glove structure, only 3 samples of the S11 curve from the FSS were collected for 3 distinct hand positions (bent middle finger, bent index finger, and hand in a horizontal position). These data underwent various image processing techniques to expand the database, resulting in a dataset composed of 1540 images. The preliminary results of this research indicate an accuracy rate of 95% for the architecture used. This architecture was defined through the training and testing of the model with different configurations in terms of the number of convolutional layers and different dimensions of the convolutional layer filters, which pointed to the best accuracy result in the 3D convolutional layers architecture with a 3x3 filter. This demonstrates the effectiveness of the neural network in identifying and classifying patterns from the S11 graph, even under challenging conditions of a predominantly augmented data-generated database. The approach used in this thesis presents the potential for a machine learning model suitable for the development of a computer-integrated wearable device for ASL sign recognition.