ARTIGO TÉCNICO Paulo Rogério de Almeida Ribeiro1, Fernando Ribeiro2 and Gil Lopes3
VISION AND DISTANCE INTEGRATED SENSOR (KINECT) FOR AN AUTONOMOUS ROBOT Abstract—This work presents an application of the Microsoft Kinect camera for an autonomous mobile robot. In order to drive autonomously one main issue is the ability to recognize signalling panels positioned overhead. The Kinect camera can be applied in this task due to its double integrated sensor, namely vision and distance. The vision sensor is used to perceive the signalling panel, while the distance sensor is applied as a segmentation filter, by eliminating pixels by their depth in the object’s background. The approach adopted to perceive the symbol from the signalling panel consists in: a) applying the depth image filter from the Kinect camera; b) applying Morphological Operators to segment the image; c) a classification is carried out with an Artificial Neural Network and a simple Multilayer Perceptron network that can correctly classify the image. This work explores the Kinect camera depth sensor and hence this filter avoids heavy computational algorithms to search for the correct location of the signalling panels. It simplifies the next tasks of image segmentation and classification. A mobile autonomous robot using this camera was used to recognize the signalling panels on a competition track of the Portuguese Robotics Open. Index Terms—Kinect, Sensors, Autonomous robot.
I. INTRODUCTION THERE is a huge variety of sensors such as temperature, strength, distance, sound and many others sensors that are employed in different applications. Robotics is one field which uses them extensively [26]. Darpa Grand Challenge [9] is one example involving many sensors of different kinds. In this task, several autonomous cars must drive itself around 240 Km on the desert, and more recently the urban challenge was performed on a city environment (a desert military city). The information obtained from the sensors is the only resource used to define the tasks, for example, to change the velocity or direction.
autonomously in a track like a traffic road. The main compulsory challenge of this robot consists of being autonomous, following the track and its two tight curves without touching the outside white lines, avoiding collisions with the tunnel, be aware of the roadwork cones, stopping on the zebra crossing, obey to the traffic light and signalling panel, etc. On this context, one can show the Formula UM, see Figure 1, type autonomous robot developed in the lab, which already participates on this competition since 2009.
The Kinect camera was designed and developed by Microsoft [1] to be used by the video game console XBOX 360. The visionary idea was to develop a new way to play video games, enabling an interaction with the player without the traditional control joystick. Kinect was first planned to be a good video game tool, but soon it was found out that it could be used as a distance and vision sensor for robotic purposes. The Portuguese Robotics Open [2] hosts robotics competitions, demos and a scientific meeting (International Conference on Mobile Robots and Competitions). One of these competitions is the autonomous driving league, where robots with maximum dimensions of 60x100x80 cm must drive
1 Paulo Ribeiro is master student of Mechatronics Engineering: University of Minho - Portugal. paulorogeriocp@gmail.com 2 Fernando Ribeiro, Associate Professor, Departamento de Electrónica Industrial - Robotics Group - Algoritmi Centre - University of Minho, Campus de Azur´em, 4800-058, Guimarães, Portugal. fernando@dei.uminho.pt 3 Gil Lopes, Auxiliar Researcher, Departamento de Electrónica Industrial - Robotics Group - Algoritmi Centre - University of Minho, Campus de Azurém, 4800-058, Guimarães, Portugal. gil@dei.uminho.pt
[8]
robótica
Figure 1 · Formula UM type autonomous robot.
For the Portuguese Robotics Open - 2011 edition, the Minho Team decided to include one innovation and proposed to apply the Kinect camera as a distance and vision sensor. This camera can be used to filter the track signalling panels. The depth image obtained with this camera is divided into planes, and these contain near and far objects. These planes can be calibrated by the user and the threshold values render them. This filter provides an elimination of some undesired objects on the track using their distances, and only when the signalling panel is between the planes further steps