External validation of a clinical instrument for real time prediction of adverse drug reactions in patients hospitalized in general wards (HARPS)
adverse drug reactions; risk prediction; external validation; hospitalized patients; clinical pharmacy; cohort study.
An adverse drug reaction (ADR) is a harmful and unintended response to a medicinal product that occurs at doses normally used in humans for prophylaxis, diagnosis, treatment of diseases, or modification of physiological function. In hospital settings, ADR risk stratification and prediction tools have the potential to optimize patient care, enhance patient safety, and reduce healthcare costs. Such tools should demonstrate clinical applicability, as well as adequate sensitivity and specificity for ADR occurrence. However, these characteristics must be confirmed through validation studies, which are more robust when conducted in multicenter settings. This study aimed to externally validate the Hospital ADR Risk Prediction Score (HARPS), a clinical tool developed to predict the immediate risk of ADRs at any point during hospitalization using data collected at the time of assessment rather than at admission. In the original HARPS derivation and internal validation study, the area under the receiver operating characteristic curve (AUC) was 0.76. This multicenter, prospective, open-cohort study was conducted in six tertiary public hospitals in Northeastern Brazil. On a weekly basis, hospitalized patients aged ≥18 years admitted to general medical and surgical wards were randomly selected from all inpatients. The HARPS score was calculated at the time of selection, and active surveillance for ADRs was performed during the subsequent 72 hours by clinical pharmacists using the Institute for Healthcare Improvement (IHI) Global Trigger Tool, interviews with healthcare professionals and patients, and medical record review. Causality assessment was performed using the Liverpool ADR Causality Assessment Tool. Two HARPS versions were evaluated: one requiring complete laboratory data (HARPS-F) and a modified version assigning a value of zero to missing laboratory variables (HARPS-L). Performance metrics included sensitivity, specificity, positive and negative predictive values, AUC, and F1 score. Among 820 patients with complete follow-up (mean age 54.1 ± 17 years; 54% male), the cumulative incidence of ADRs was 14.7%. For HARPS-F, sensitivity was 44.1%, specificity 63.3%, positive predictive value 23.4%, negative predictive value 81.7%, AUC 0.63 (95% CI: 0.57–0.68), and F1 score 0.31. For HARPS-L, sensitivity was 37.6%, specificity 72.3%, positive predictive value 22.0%, negative predictive value 84.8%, AUC 0.63 (95% CI: 0.58–0.68), and F1 score 0.28. This external validation did not confirm the performance of HARPS observed in the original internal validation study. Nevertheless, HARPS remains the only tool specifically designed for dynamic and continuous assessment of ADR risk throughout hospitalization. Its performance, comparable to that of tools applied only at hospital admission, demonstrates the feasibility of this methodological approach and highlights the need for further refinement.