Identification of high-risk patients for referral through machine learning assisting the decision making to manage minor ailments in community pharmacies

Abstract

Background: Data analysis techniques such as machine learning have been used for assisting in triage and the diagnosis of health problems. Nevertheless, it has not been used yet to assist community pharmacists with services such as the Minor Ailment Services These services have been implemented to reduce the burden of primary care consultations in general medical practitioners (GPs) and to allow a better utilization of community pharmacists’ skills. However, there is a need to refer high-risk patients to GPs. Aim: To develop a predictive model for high-risk patients that need referral assisting community pharmacists’ triage through a minor ailment service. Method: An ongoing pragmatic type 3 effectiveness-implementation hybrid study was undertaken at a national level in Spanish community pharmacies since October 2020. Pharmacists recruited patients presenting with minor ailments and followed them 10 days after the consultation. The main outcome measured was appropriate medical referral (in accordance with previously co-designed protocols). Nine machine learning models were tested (three statistical, three black box and three tree models) to assist pharmacists in the detection of high-risk individuals in need of referral. Results: Over 14′000 patients were included in the study. Most patients were female (68.1%). With no previous treatment for the specific minor ailment (68.0%) presented. A percentage of patients had referral criteria (13.8%) however, not all of these patients were referred by the pharmacist to the GP (8.5%). The pharmacists were using their clinical expertise not to refer these patients. The primary prediction model was the radial support vector machine (RSVM) with an accuracy of 0.934 (CI95 = [0.926,0.942]), Cohen’s kappa of 0.630, recall equal to 0.975 and an area under the curve of 0.897. Twenty variables (out of 61 evaluated) were included in the model. radial support vector machine could predict 95.2% of the true negatives and 74.8% of the true positives. When evaluating the performance for the 25 patient’s profiles most frequent in the study, the model was considered appropriate for 56% of them. Conclusion: A RSVM model was obtained to assist in the differentiation of patients that can be managed in community pharmacy from those who are at risk and should be evaluated by GPs. This tool potentially increases patients’ safety by increasing pharmacists’ ability to differentiate minor ailments from other medical conditions