Exploring the Use of Projecting Conflicting Gradients in Multi-task Neural Networks with an Application to Amyotrophic Lateral Sclerosis

In medicine, it is often necessary to evaluate multiple outcomes at the same time to capture the complexity of patient conditions. Multi–task learning (MTL), a machine learning framework that addresses multiple tasks using a single model, has emerged as a promising approach for such problems. This framework stands to offer the dual benefits of resource efficiency and improved generalisation performance across tasks. However, the multi–task setting introduces several optimisation challenges, primarily due to gradient interference between tasks, i.e., when the gradients computed for different tasks conflict with each other during the optimisation process. To address this, gradient manipulation strategies, such as Projecting Conflicting Gradients (PCGrad), have been developed to mitigate these issues, reducing interference by projecting gradients of different objective functions into non–conflicting directions. Although the use of MTL and gradient manipulation techniques is very popular and continually evolving in fields like computer vision and natural language processing, their application in medicine remains limited. In light of this, in the present work, we investigate the effectiveness of PCGrad in improving MTL model performance in a multi–outcome setting with three classification tasks and one regression task in patients with amyotrophic lateral sclerosis (ALS). We show that using PCGrad improved performance on all considered tasks, especially in the regression one, with a statistically significant reduction in the mean absolute error of 5%.