Psychiatrists make life-changing decisions every day. Should a patient be discharged or is the patient at high risk of committing suicide? Will coercion be necessary during the hospitalization of a new patient? Decisions like these have vital consequences for the lives and experiences of both patients and staff but they are made under increasing time and resource constraints. Currently, psychiatrists are assisted in their decision-making by studies of risk factors based on retrospective statistical analyses, but increasing evidence suggests that machine learning might be superior to these methods in terms of assessing risk and predicting outcomes.
Using machine learning within precision psychiatry has the potential of building objective algorithm-based frameworks with individual treatment-response prediction across a diversity of psychiatric conditions. It recently became possible to implement predictive machine learning models directly in the Danish health care system. This opens up the possibility of implementing automated models capable of flagging patients at increased risk of e.g., suicide or coercion, thus enabling clinicians to take preventive measures to avoid these outcomes.
For these reasons, predictive models based on machine learning are in high demand. However, many challenges still remain before this vast potential can be realized in the health care sector. Modeling and evaluation is challenged by the fact that the relevant outcomes are rare occurrences, meaning that data is highly imbalanced.
Imbalanced data is challenging because machine learning models and evaluation metrics assume that the class distribution is balanced and that misclassification errors are equal across classes. This leads the models to be biased towards the majority class and thus underpredicting the relevant outcomes. Many strategies and specific algorithms have been developed to address the shortcomings of imbalanced learning, but the performance seems to differ significantly between data sets. This illustrates the need for a more general framework as well as performance metrics, where hyper parameters can be tuned to consider e.g., the degree of imbalance in the data. Furthermore, such a unified framework should consider the proper strategies both for model training and model assessment.
The objective of this research project is to address the limitations of the existing predictive models in psychiatry by exploring a general framework for the modeling and evaluation of machine learning models based on imbalanced data. The framework will be of use to a wide range of AI applications in psychiatry, and the research project will illustrate this by developing specific models capable of predicting suicide and coercion.