Using VLAD scores to have a look insight ICU performance: towards a modelling of the errors

RATIONALE, AIMS AND OBJECTIVES: Mortality prediction models using logistic regression analysis play a pivotal role in intensive care quality evaluation, allowing a hospital's performance to be compared with a standard. However, when a difference between predicted and observed mortality exists, that is, the numerator of the Variable Life Adjusted Display (VLAD) score, the investigation for a possible explanation could be arduous. In this article we tested the ability of Bayesian Network (BN) to identify factors determining the negative discrepancy between expected and actual outcomes recorded in four Italian intensive care units (ICUs). METHODS: A BN was implemented to predict the extent of the expected-observed distance quantified by the VLAD score. BN performance was compared with those of a set of tools including Linear Model, Random Forest Regression Tree analysis, Artificial Neural Networks and Support Vector Machine. RESULTS: BN allows the identification of critical areas responsible for bad performance. Compared with other techniques, BN always explains a higher variance percentage and it shows similar or superior discrimination ability. CONCLUSIONS: BN, being able to guide interpretation of covariates role by means of a graphic representation of relationships, confirms its utility particularly where many interactions between predictors exist and when a coherent set of theories regarding which variables are related and how is not available.