Models based on trasformer architecture have shown promising results in areas of study such as natural language processing (NLP). In this same area of study, chatbots are tools widely used in customer service tasks. This work will use RASA and its DIET classifier to perform a comparative study of the impact of pre-trained trasformer models in the development of a chatbot. As a scenario for this study, the case of the Court of Auditors of Rio Grande do Norte (TCE/RN) and its need to implement a chatbot aimed at improving the efficiency of its public service sector was analyzed.