Logotype Mälardalen University

MRTC research projects



Two Camera System

Leader: Lars Asplund
Members: Lars Asplund, Giacomo Spampinato, Jörgen Lidholm, Fredrik Ekstrand, Carl Ahlberg
Former: Karl Ingström
Research group:Robotics
Keywords: FPGA Vision robotics
Status: finished , start date: 2008-04-01 , End date: 2011-03-30
Partners: The industrial partners are Hectronic, Sensor Control and MEEQ.
Funding: Knowledge Foundation. ´KKS_Logo´
Web: Project web page

 

Overview

In robotics it is essential to have adequate sensors, and one of the most important sensors for a robot is the vision sensor. When it comes to vision sensors, speed and size is of great importance. Vision applications using standard techniques are widely used in the industry today, although these systems are bulky and slow. The latest development of reconfigurable hardware together with new sensor chips enable a new type of high performance vision systems to be built. The research question that needs to be solved in order for this new type of vision systems to become useful for robotic applications (industrial robots or autonomous service robots) is how to use vision algorithms in reconfigurable hardware.

The current approach taken in this work is to find algorithms that work on a continuous data flow, using sliding windows, and thus eliminating the need for external memory for storing a complete image.
 

Project publications


GIMME - A General Image Multiview Manipulation Engine , Carl Ahlberg, Jörgen Lidholm, Fredrik Ekstrand, Giacomo Spampinato, Mikael Ekström, Lars Asplund, Proceedings of the International Conference on ReConFigurable Computing and FPGAs (ReConFig 2011), Cancun, November, 2011

Resource Optimized Stereo Matching in Reconfigurable Hardware for Autonomous Systems, Fredrik Ekstrand, Licentiate Thesis, Mälardalen University, Västerås, Sweden, September, 2011
 

Results achieved

GIMME - A General Image Multiview Manipulation Engine

GIMME is the hardware platform that emerged from the Two-Camera System collboration. It is a highly flexible reconfigurable stand-alone mobile two-camera vision platform with stereo-vision capability. GIMME relies on reconfigurable hardware (FPGA) to perform application-specific low to medium-level image-processing. The Qseven-extension enables additional processing power. Thanks to its compact design, low power consumption, standardized interfaces (power and communication) GIMME is an ideal vision platform for autonomous and mobile robot applications. ´´ GIMME features two 5-megapixel CMOS sensors, an FPGA (Spartan-3A DSP 1800), 32MB SDRAM, 16MB flash memory, a Qseven interface, 100Mbit Ethernet and USB communication. ´gimme_system´ Streaming of images via Ethernet is an important test as it involves acquiring and forwarding sensor information, and hence requires a framework of IP-blocks for handling the communication between the FPGA and other hardware components. 5.5 fps is achievable when streaming 640x480 RGB-images of 12-bit color-depth, using a Ethernet frame-based protocol. ´streamed_image´ The main idea behind the hardware design is to implement image processing algorithms within the FPGA. This in order to extract important information from the image data stream. In the Malta project features from the Stephen-Harris combined corner and edge detector are used for autonomous navigation. The frame rate is 44.5 fps on intensity-based images from a 640x480 RGB-image stream of 12-bit color depth. ´streamed_harris´
 

Future work

The project continues in an EU-project.


  • Mälardalen University |
  • Box 883 |
  • 721 23 Västerås/Eskilstuna |
  • 021-101300, 016-153600 |
  • webmaster |
  • Latest update: 2010.02.23