Banca de DEFESA: WALLINSON DE LIMA SILVA

Uma banca de DEFESA de MESTRADO foi cadastrada pelo programa.
STUDENT : WALLINSON DE LIMA SILVA
DATE: 26/07/2022
TIME: 13:00
LOCAL: Remoto e Online - Google Meet
TITLE:

Using Metamorphic Tests to Verify a Chatbot Application

KEY WORDS:

Metamorphic Testing; Software Testing on Chatbots; Test Case Generation.

PAGES: 73
BIG AREA: Ciências Exatas e da Terra
AREA: Ciência da Computação
SUBÁREA: Metodologia e Técnicas da Computação
SPECIALTY: Engenharia de Software
SUMMARY:

Chatbots are software that make use of Artificial Intelligence (IA) to provide interaction between humans and machines through Natural Language (NL), resembling a conversation between two people. It is a current topic that has received a lot of attention in recent years. However, this type of application presents a behavior that is sometimes unpredictable, as different users can request the same set of actions in different ways and the application can return something different to each of the users. This makes the software testing activity somewhat complex, since it is not simple to define the test oracle. The test oracle is the process that verifies the test result, judging the response of the application, program or function as correct or incorrect through specification and planning. In this scenario, an appropriate testing technique to deal with the oracle problem is Metamorphic Testing (MT). This technique was proposed as a solution to minimize the oracle definition problem, since the application response verification process is performed by the definition of Metamorphic Relations (MRs), elements that assume the role of the oracle. Metamorphic relationships are properties of the test object application that define the behavior that the application must maintain when submitted to a given test case. However, despite Metamorphic Testing being a well-established technique to deal with the problem of the lack of a test oracle, it is still little explored in the context of chatbots. Thus, the objective of this work was to investigate the applicability and effectiveness of the MT technique in a chatbot application developed by a public company. Therefore, it was necessary to develop a tool to automate the test process by applying the MT concepts. For the evaluation of the technique we selected 22 Source Test Cases (STCs) divided into 3 Intents and input this information into the automation tool to run the tests of the 20 Metamorphic Relations. Through the execution of the tool, it was possible to generate more than 4100 Follow-up Test Cases (FTCs), derived from the STCs. The verification took place in two test batteries using two versions of the same application with 3 months of development difference. With the application of the MT technique, it was observed that 25% of the FTCs revealed defects, representing more than 900 disapproved FTCs, per version tested.

COMMITTEE MEMBERS:
Presidente - 2274774 - EIJI ADACHI MEDEIROS BARBOSA
Externa ao Programa - 1709820 - ROBERTA DE SOUZA COELHO - UFRNExterno à Instituição - FABIANO CUTIGI FERRARI - UFSCAR