Comparison of threshold tuning methods for predictive monitoring

Predictive monitoring techniques produce signals in case of a high predicted probability of an undesirable event, such as mortality, heart attacks, or machine failure. When using these predicted probabilities to classify the unknown outcome, a decision threshold needs to be chosen in statistical and machine learning models. In many cases, this is set to 0.5 by default. However, this may not lead to an acceptable model performance. To mitigate this issue, different threshold optimization approaches have been proposed in the literature. In this paper, we compare existing thresholding techniques to achieve a desired false alarm rate, and also evaluate the corresponding precision and recall performance metrics. A simulation study is conducted and a real-world example on a medical dataset is provided.