Kernel-based learning of orthogonal functions

The paper deals with the reconstruction of functions from sparse and noisy data in suitable intersections of Hilbert spaces that account for orthogonality constraints. Such problem is becoming more and more relevant in several areas like imaging, dictionary learning, compressed sensing. We propose a new approach where it is interpreted as a particular kernel-based multi-task learning problem, with regularization formulated in a reproducing kernel Hilbert space. Special penalty terms are then designed to induce orthogonality. We show that the problem can be given a Bayesian interpretation. This then permits to overcome nonconvexity through a novel Markov chain Monte Carlo scheme able to recover the posterior of the unknown functions and also to understand from data if the orthogonal constraints really hold.