Multimodal robot perception for robust human tracking and recognition

Modern service robot must be provided with a fast, reliable system for tracking and recognizing people in order to operate safely and efficiently in human environ- ments. While most of the current approaches consider these two tasks as independent processes, the solution presented in this chapter is based on an original multimodal system that performs simultaneous people tracking and identification, combining dif- ferent sensors to detect humans as well as algorithms to recognize them. The multisensor solution adopted for human detection, based on the robot's laser and camera, is initially introduced. The laser device can detect human legs, while the camera locates frontal faces. Thanks to a robust and efficient histogram comparison, vision is also used to distinguish the clothes of the subjects being tracked. Sensor in- formation is fused within a Bayesian framework to perform joint people tracking and recognition. The solution is based on a bank of filters that integrates all the available observations and generates estimates weighted by identity's probabilities. The infor- mation for human height, clothes and face recognition is stored inside a database of known people. The modularity of the design facilitates the integration of additional perception algorithms (e.g. sound localization, voice recognition, etc.) for possible improvements of the robot' sensing system. The effectiveness of the current approach is demonstrated by several experiments conducted with real mobile robots in presence of people. The successful performance of the proposed solution confirms also its high potential for real world applications of service robotics.