Fast and accurate crop and weed identification with summarized train sets for precision agriculture

In this paper we present a perception system for agriculture robotics that enables an unmanned ground vehicle (UGV) equipped with a multi spectral camera to automatically perform the crop/weed detec- tion and classification tasks in real-time. Our approach exploits a pipeline that includes two different convolutional neural networks (CNNs) applied to the input RGB+near infra-red (NIR) images. A lightweight CNN is used to perform a fast and robust, pixel- wise, binary image segmentation, in order to extract the pixels that rep- resent projections of 3D points that belong to green vegetation. A deeper CNN is then used to classify the extracted pixels between the crop and weed classes. A further important contribution of this work is a novel unsupervised dataset summarization algorithm that automatically selects from a large dataset the most informative subsets that better describe the original one. This enables to streamline and speed-up the manual dataset label- ing process, otherwise extremely time consuming, while preserving good classification performances. Experiments performed on different datasets taken from a real farm robot confirm the effectiveness of our approach.