Depth-based frontal view generation for pose invariant face recognition with consumer RGB-D sensors

In this work, we propose to exploit depth information to build a pose-invariant face recognition algorithm from RGB-D data. Our approach first estimates the head pose and then generates a frontal view for those faces that are rotated with respect to the frame of the camera. Then, some interest points of the face are detected by means of a Random Forest applied to the RGB image and they are used as keypoints where to compute feature descriptors. Around these points and their 3D counterpart, we extract both 2D and 3D local descriptors, which are then concatenated and classified by means of a Support Vector Machine trained in “one-versus-all” fashion. In order to validate the accuracy of the system with data from consumer RGB-D sensors, we created the IAS-Lab RGB-D Face Dataset, a new public dataset in which RGB-D data are acquired with a second generation Microsoft Kinect. The reported experiments show that the depth-aided approach we propose allows to improve the recognition rate up to 50 %.