Using Metamorphic Tests to Verify a Chatbot Application
Metamorphic Tests; Software Testing on Chatbots; Generation of Sentences.
Chatbots are software that make use of Artificial Intelligence (AI) to provide interaction between humans and machines through Natural Language (NL), similar to a conversation between two people. It is a current topic and one that has received a lot of attention in recent years. However, this type of application presents a behavior that is sometimes unpredictable, as different users can request the same set of actions in different ways and the application can return something different for each one of the users. This makes the software testing activity complex, as it is not simple to define the testing oracle. The test oracle is the process that verifies the test result, judging the response of the application, program or function as correct or incorrect upon specification and planning. In this scenario, an appropriate testing technique to deal with the oracle problem is the Metamorphic Test (MT). This technique was proposed as a solution to minimize the oracle definition problem, as the application response validation process is done by defining Metamorphic Relationships (MRs), elements that assume the role of the oracle. Metamorphic relationships are properties of the test object application that define the behavior that the application must maintain when submitted to a given test case. However, while Metamorphic Tests is a well-established technique for dealing with the problem of the lack of a testing oracle, it is still underexplored in the context of chatbots. Thus, our objective of this work is to investigate the applicability and effectiveness of TM in chatbot applications developed by Dataprev, and to establish a test strategy to compose the development lifecycle of these applications.