Banca de QUALIFICAÇÃO: ESMERALDINA LOPES GONCALVES

Uma banca de QUALIFICAÇÃO de MESTRADO foi cadastrada pelo programa.
STUDENT : ESMERALDINA LOPES GONCALVES
DATE: 08/07/2025
TIME: 09:00
LOCAL: Videoconferência
TITLE:

Validation of the Quality of ChatGPT's Responses to Frequently Asked Questions About Gestational and Congenital Syphilis

KEY WORDS:

artificial intelligence; frequently asked questions (FAQ); gestational syphilis; congenital syphilis.

PAGES: 80
BIG AREA: Ciências da Saúde
AREA: Saúde Coletiva
SUMMARY:

Data from the Brazilian Ministry of Health indicate an increase in the detection rate of acquired syphilis and the incidence of congenital syphilis, associated with several factors, including misinformation. ChatGPT is a generative artificial intelligence (AI) chatbot that uses natural language processing and can answer a wide range of questions, including those related to syphilis. This dissertation aims to validate the quality of ChatGPT's responses to frequently asked questions about gestational and congenital syphilis. This is a descriptive, cross-sectional, observational validation study with a quantitative approach. The dissertation followed these steps: 1. Identification and selection of frequently asked questions about gestational and congenital syphilis; 2. Generation of responses using ChatGPT 4.0; 3. Validation of the responses by healthcare professionals and the target audience; 4. Evaluation of the readability of ChatGPT’s responses. During the validation stage, the judges included both healthcare professionals and lay individuals. The professional group was composed of physicians, nurses, and pharmacists involved in the treatment of gestational and congenital syphilis. The lay group consisted of literate pregnant women, aged 18 or older, without a syphilis diagnosis, who were receiving care at a public maternity hospital in the Northern Region of Brazil. The judges evaluated the responses based on criteria of clarity, relevance, accuracy, comprehensiveness, and overall satisfaction (perceived value of the information provided). Accuracy and comprehensiveness were not assessed by the lay judges. After agreeing to participate and signing the informed consent form, participants received a Google Forms link to complete the evaluation. Data analysis included calculation of the Content Validity Index (CVI) and Cronbach’s alpha coefficient. Text readability was assessed using the ALT software, applying various readability formulas, including Flesch, Gulpease, Flesch-Kincaid, Gunning Fog, ARI, Coleman-Liau, and a General Index. To analyze the readability results, analysis of variance (ANOVA) was conducted across three versions of responses, along with Pearson correlation tests between the readability indices. IBM SPSS Statistics® version 22 and Jamovi version 2.6 were used for statistical analyses. The validation results indicated high acceptance of ChatGPT’s responses by the professional audience, with particularly high scores for clarity (CVI = 0.94) and relevance (CVI = 0.91). Among the pregnant participants, the scores were also satisfactory, with mean ratings above 4.0 on a 5-point Likert scale. Internal consistency was considered high across all criteria, as indicated by Cronbach’s alpha. Regarding readability, most responses were found to have a medium or easy reading level, appropriate for the target audience. It is concluded that ChatGPT has potential as a complementary tool for health education, especially in contexts of social vulnerability.

COMMITTEE MEMBERS:
Presidente - 2665457 - GIDYENNE CHRISTINE BANDEIRA SILVA DE MEDEIROS
Interno - 1149540 - ANGELO GIUSEPPE RONCALLI DA COSTA OLIVEIRA
Externa ao Programa - 2166798 - ADALA NAYANA DE SOUSA MATA - UFRN