A superpixel segment is a group of pixels that carry similar information. The Simple Linear Iterative Clustering (SLIC) is a well-known algorithm for generating superpixels that offers a good balance between accuracy and efficiency. Nevertheless, due to its high computational requirements, the algorithm does not meet the demands of real-time embedded applications in terms of speed and resources. This paper proposes a fully-pipelined FPGA architecture based on SLIC, dubbed FP-SLIC, that exhibits 1) a simplified and efficient algorithm of reduced computational complexity that facilitates algorithm development for FPGAs, 2) a fully pipelined FPGA design operating at 40MHz with a throughput of one pixel per cycle, and 3) a memory-efficient architecture that eliminates the requirement for external memory. FP-SLIC shows promising BSDS500 benchmark results, especially considering boundary recall for less than 1000 superpixels, where it performs better than related works, while, at the same time, accomplishing a throughput of 259 frames per second (fps).

Emerging precision agriculture techniques rely on the frequent collection of high-quality data which can be acquired efficiently by unmanned aerial systems (UAS). The main obstacle for wider adoption of this technology is related to UAS operational costs. The path forward requires a high degree of autonomy and integration of the UAS and other cyber physical systems on the farm into a common Farm Management System (FMS) to facilitate the use of big data and artificial intelligence (AI) techniques for decision support. Such a solution has been implemented in the EU project AFarCloud (Aggregated Farming in the Cloud). The regulation of UAS operations is another important factor that impacts the adoption rate of agricultural UAS. An analysis of the new European UAS regulations relevant for autonomous operation is included. Autonomous UAS operation through the AFarCloud FMS solution has been demonstrated at several test farms in multiple European countries. Novel applications have been developed, such as the retrieval of data from remote field sensors using UAS and in situ measurements using dedicated UAS payloads designed for physical contact with the environment. The main findings include that (1) autonomous UAS operation in the agricultural sector is feasible once the regulations allow this; (2) the UAS should be integrated with the FMS and include autonomous data processing and charging functionality to offer a practical solution; and (3) several applications beyond just asset monitoring are relevant for the UAS and will help to justify the cost of this equipment.

Stereo correspondence is a well-established research topic and has spawned categories of algorithms combining several processing steps and strategies. One core part to stereo correspondence is to determine matching cost between the two images, or patches from the two images. Over the years several different cost metrics have been proposed, one being the Census Transform (CT). The CT is well proven for its robust matching, especially along object boundaries, with respect to outliers and radiometric differences. The CT also comes at a low computational cost and is suitable for hardware implementation. Two key developments to the CT are non-centric and sparse comparison schemas, to increase matching performance and/or save computational resources. Recent CT algorithms share both traits but are handcrafted, bounded with respect to symmetry, edge lengths and defined for a specific window size. To overcome this, a Genetic Algorithm (GA) was applied to the CT, proposing the Genetic Algorithm Census Transform (GACT), to automatically derive comparison schemas from example data. In this paper, FPGA-based hardware acceleration of GACT, has enabled evaluation of census windows of different size and shape, by significantly reducing processing time associated with training. The experiments show that lateral GACT windows produce better matching accuracy and require less resources when compared to square windows.

Autonomous agents rely on information from the surrounding environment to act upon. In the array of sensors available, the image sensor is perhaps the most versatile, allowing for detection of colo ...

The Census Transform (CT) is a well proven method for stereo vision that provides robust matching, with respect to object boundaries, outliers and radiometric distortion, at a low computational cost. Recent CT methods propose patterns for pixel comparison and sparsity, to increase matching accuracy and reduce resource requirements. However, these methods are bounded with respect to symmetry and/or edge length. In this paper, a Genetic algorithm (GA) is applied to find a new and powerful CT method. The proposed method, Genetic Algorithm Census Transform (GACT), is compared with the established CT methods, showing better results for benchmarking datasets. Additional experiments have been performed to study the search space and the correlation between training and evaluation data.

The process of combining two views of a scene in order to obtain depth information is called stereo vision. When the same is done using a computer it is then called computer stereo vision. Stereo vision is used in robotic application where depth of an object plays a role. Two cameras mounted on a rig is called a stereo camera system. Such a system is able to capture two views and enable robotic application to use the depth information to complete tasks. Anomalies are bound to occur in such a stereo rig, when both the cameras are not parallel to each other. Mounting of the cameras on a rig accurately has physical alignment limitations. Images taken from such a rig has inaccurate depth information and has to be rectified. Therefore rectification is a pre-requisite to computer stereo vision. One such a stereo rig used in this thesis is the GIMME2 stereo camera system. The system has two 10 mega-pixel cameras with on-board FPGA, RAM, processor running Linux operating system, multiple Ethernet ports and an SD card feature amongst others. Stereo rectification on memory constrained hardware is a challenging task as the process itself requires both the images to be stored in the memory. The FPGA on the GIMME2 systems must be used in order to achieve the best possible speed. Programming a system that does not have a display and for used for a specific purpose is called embedded programming. The purpose of this system is distance estimation and working with such a system falls in the Embedded Systems program. This thesis presents a method that makes rectification a step ahead for this particular system. The functionality of the algorithm is shown in MATLAB and using VHDL and is compared to available tools and systems.

This paper presents GIMME2, an embedded stereovision system, designed to be compact, power efficient, cost effective, and high performing in the area of image processing. GIMME2 features two 10 megapixel image sensors and a Xilinx Zynq, which combines FPGA-fabric with a dual-core ARM CPU on a single chip. This enables GIMME2 to process video-rate megapixel image streams at real-time, exploiting the benefits of heterogeneous processing.

This paper proposes a segmentation-based approach for matching of high-resolution stereo images in real time. The approach employs direct region matching in a raster scan fashion influenced by scanline approaches, but with pixel decoupling. To enable real-time performance it is implemented as a heterogeneous system of an FPGA and a sequential processor. Additionally, the approach is designed for low resource usage in order to qualify as part of unified image processing in an embedded system.

This paper proposes an approach to image processing for high performance vision systems. Focus is on achieving a scalable method for real-time disparity estimation which can support high resolution images and large disparity ranges. The presented implementation is a non-local matching approach building on the innate qualities of the processing platform which, through utilization of a heterogeneous system, combines low-complexity approaches into performing a high-complexity task. The complementary platform composition allows for the FPGA to reduce the amount of data to the CPU while at the same time promoting the available informational content, thus both reducing the workload as well as raising the level of abstraction. Together with the low resource utilization, this allows for the approach to be designed to support advanced functionality in order to qualify as part of unified image processing in an embedded system.

The Black Pearl is a custom made autonomous underwater vehicle developed at Malardalen University, Sweden. It is built in a modular fashion, including its mechanics, electronics and software. After a successful participation at the RoboSub competition in 2012 and winning the prize for best craftsmanship, this year we made minor improvements to the hardware, while the focus of the robot's evolution shifted to the software part. In this paper we give an overview of how the Black Pearl is built, both from the hardware and software point of view.

This paper presents an embedded vision system based on reconfigurable hardware (FPGA) to perform stereo image processing and 3D mapping of sparse features for autonomous navigation and obstacle detection in industrial settings. We propose an EKF based visual SLAM to achieve a 6D localization of the vehicle even in non flat scenarios. The system uses vision as the only source of information. As a consequence, it operates regardless of the odometry from the vehicle since visual odometry is used.

There is a need for compact, high-speed, and low-power vision systems for enabling real-time mobile autonomous applications. The best approach to achieve this is to implement the bulk of the application in hardware. Reconfigurable hardware meet these requirements without the limitation of fixed functionality that accompanies application-specific circuits. Resource constraints of reconfigurable hardware calls for optimized implementations i terms of resource usage with maintained performance. The research group in Robotics at Malardalen University is moving toward the completion of a reconfigurable hardware-platform for stereo vision, coupled with a compact embedded computer. This system will incorporate hardware-based preprocessing components enabling visual perception for autonomous machines. This thesis covers the reconfigurable hardware section of the vision system concerning the realization of scene depth extraction. It shows the advantages of image preprocessing in hardware and propose a resource optimized approach to stereo matching. The work quantifies the impact of reduced resource utilization and a desire for increased accuracy in disparity estimation. The implemented stereo matching approach performs on par with recent similar implementations in terms of accuracy, but excels in terms of resource utilization and resource sharing, as the external memory requirement is removed for larger images. Future work aims to further include processes for navigation, and structure and object recognition. Furthermore, the system will be adapted to real world scenarios, both indoors and outdoors.

This paper presents GIMME (General Image Multiview Manipulation Engine), a highly flexible reconfigurable standalone mobile two-camera vision platform with stereo-vision capability. GIMME relies on reconfigurable hardware (FPGA) to perform application-specific low to medium-level imageprocessing at video-rate. A high-speed stereo-matching approach combining segmentation and 1D SAD for high accuracy and unlimited disparity range at minimal resource utilization is presented. Thanks to its compact design, low power consumption and standardized interfaces (power and communication), GIMME is an ideal vision platform for autonomous and mobile robot applications.

This paper proposes a 1-dimensional implementation of area-based stereo matching with minimal resource utilization. It achieves an acceptable disparity map without the use of expensive resources. The matching accuracy for the approach can in some extent even outperform that of its 2-dimensional counterpart. Additionally, as it excels in terms of frame rate and resource utilization, it is highly suitable for real-time stereo-vision systems.

The research presented addresses the emerging topic of AGVs (Automated Guided Vehicles) specifically related to industrial sites. The work presented has been carried out in the frame of the MALTA project (Multiple Autonomous forklifts for Loading and Transportation Applications), a joint research project between industry and university, funded by the European Regional Development and Robotdalen, in partnership with theSwedish Knowledge Foundation. The project objective is to create fully autonomous forklift trucks for paper reel handling. The result is expected to be of general benefit for industries that use forklift trucks in their material handling through higher operating efficiency and better flexibility with reduced risk for accidents and handling damages than if only manual forklift trucks are used. A brief overview of the state of the art in AGVs will be reported in order to better understand the new challenges and technologies. Among the emerging technologies used for vehicle automation, vision is one of the most promising in terms of versatility and efficiency, with a high potential to drastically reduce the costs.

This paper presents GIMME (General Image Multiview Manipulation Engine), a highly flexible reconfigurable stand-alone mobile two-camera vision platform with stereo-vision capability. GIMME relies on reconfigurable hard-ware (FPGA) to perform application-specific low to medium-level image-processing at video-rate. The Qseven-extension enables additional processing power. Thanks to its compact design, low power consumption and standardized interfaces (power and communication), GIMME is an ideal vision platform for autonomous and mobile robot applications.

This paper presents an embedded vision system based on reconfigurable hardware (FPGA) and two CMOS cameras to perform stereo image processing and 3D mapping for autonomous navigation. We propose an EKF based visual SLAM and sparse feature detectors to achieve 6D localization of the vehicle in non flat scenarios. The system can operate regardless of the odometry information from the vehicle since visual odometry is used. As a result, the final system is compact and easy to install and configure.