Challenges in constructing and validating research instruments with adolescents: a warning on
the quality of data collected
“Adolescent Behavior”, “Evaluation Research”, “Reliability of Results”, “Reproducibility of Results”, “Research Methodology”, “reliability and validity”
Abstract: Purpose: To evaluate the difficulties to obtain reliable data from adolescents' self-reports by means of constructing and validating instruments capable of assessing knowledge and attitudes regarding contraception. Methods: An exploratory study with students from public schools, with 12 to 18 years old (n=123), who answer to two instruments constructed specifically for this study. The internal consistency was determined by Cronbach's α for each attribute set after factorial analysis that used the varimax rotation type. The reproducibility was assessed by Kappa test in three testing conditions, where the distinction variables were: time interval between test and retest (42 days or 4 days). Results: In the questionnaire, three factors explain 51,84% of the variation and at attitude scale, six factors explain 71,21% of the variation and in both instruments the Cronbach's alpha was above 0,70. The consistency was poor (k<0,20) for most items when the retest was after 42 days, which increased an when the interval was 4 days. In conditions of lower interval, the higher level of knowledge, the values of Kappa were above 0,20 for all items in both instruments tested. Conclusion: The consistency of responses was one of the main difficulties one faces in validating these two instruments. However, agreement was higher under conditions of shorter time intervals between the test and retest, the highest level of knowledge of participants. These results highlights the importance in the checking the accuracy of the answers provided and alert that researchers need to consider the instability proper of adolescents and their tendency to overlook approaches to which they do not identify themselves.