Evaluating propensity-score-based methods in treatment effect estimation: a simulation study

In observational studies of treatment effectiveness with real-world data, the recorded treatment assignment is not random but is influenced by external factors such as patient characteristics, reimbursement policies, and existing guidelines. Therefore, the treatment effect can only be estimated after accounting for confounding factors. Propensity score (PS) methods are a widely used family of techniques for this purpose. Although all PS methods estimate the a posteriori probability of treatment assignment given patient covariates, they approach the treatment effect estimation from different statistical perspectives, making it interesting to compare their performance. In this work, we propose a simulation experiment in which a hypothetical cohort of subjects is simulated across three scenarios with increasing complexity in the relationships between covariates and treatment. Different PS-based methods are compared in terms of their performance in estimating the treatment effect and their robustness in various scenarios, with a particular focus on the parsimony of the methods, i.e., their complexity-performance trade-off. The investigated methods are Propensity Score Matching (PSM), Inverse Probability Weighting (IPW), g-computation, and Targeted Maximum Likelihood Estimation (TMLE). All methods tested proved effective in terms of estimating the treatment effect. From a computational complexity point of view, we found that IPW is a more parsimonious technique compared to TMLE, as the latter requires significantly higher computational time, which may limit its practical applicability. PSM shows intermediate performance, providing good estimates of the treatment effects, with an average computational time between those of IPW and TMLE.