title: From prediction to treatment decision: aligning development, evaluation and monitoring of prediction models for decision support

abstract: From the Apgar score to cardiovascular risk-management: clinicians use prediction models to inform medical decision making. Though often hailed for making healthcare more personalized, decision support with prediction models carries risks as the models are typically evaluated for predictive accuracy, not impact on decision making. Through so-called ‘self-fulfilling prophecies’, I show how prediction models that appear to be predicting well before and after being deployed in practice, can actually be causing only harm along the way. Next, I describe how prediction models with explicit causal design (‘prediction-under-intervention’-models) do not suffer from several pathologies of standard prediction models, and can be developed, evaluated and monitored in a way that directly aligns with the target of decision making.

References:

1. van Amsterdam WAC, Jong PA de, Verhoeff JJC, Leiner T, Ranganath R. From algorithms to action: improving patient care requires causality. BMC Medical Informatics and Decision Making. 2024;24(1). doi:10.1186/s12911-024-02513-3
2. Van Amsterdam WAC, Van Geloven N, Krijthe JH, Ranganath R, Cinà G. When accurate prediction models yield harmful self-fulfilling prophecies. Patterns. 2025 Apr;6(4):101229. doi:10.1016/j.patter.2025.101229
3. Van Geloven N, Keogh RH, Van Amsterdam W, Cinà G, Krijthe JH, Peek N, et al. The Risks of Risk Assessment: Causal Blind Spots When Using Prediction Models for Treatment Decisions. Ann Intern Med. 2025 Sep;178(9):1326–33. doi:10.7326/ANNALS-24-00279