[PDF] Real Time Multimodal Semantic Scene Understanding For Autonomous Ugv Navigation eBook

Author : Yifei Zhang
Publisher :
Page : 114 pages
File Size : 25,37 MB
Release : 2021
Category :
ISBN :

GET BOOK

Robust semantic scene understanding is challenging due to complex object types, as well as environmental changes caused by varying illumination and weather conditions. This thesis studies the problem of deep semantic segmentation with multimodal image inputs. Multimodal images captured from various sensory modalities provide complementary information for complete scene understanding. We provided effective solutions for fully-supervised multimodal image segmentation and few-shot semantic segmentation of the outdoor road scene. Regarding the former case, we proposed a multi-level fusion network to integrate RGB and polarimetric images. A central fusion framework was also introduced to adaptively learn the joint representations of modality-specific features and reduce model uncertainty via statistical post-processing.In the case of semi-supervised semantic scene understanding, we first proposed a novel few-shot segmentation method based on the prototypical network, which employs multiscale feature enhancement and the attention mechanism. Then we extended the RGB-centric algorithms to take advantage of supplementary depth cues. Comprehensive empirical evaluations on different benchmark datasets demonstrate that all the proposed algorithms achieve superior performance in terms of accuracy as well as demonstrating the effectiveness of complementary modalities for outdoor scene understanding for autonomous navigation.

Author : Yifei Zhang
Publisher :
Page : 0 pages
File Size : 49,55 MB
Release : 2021
Category :
ISBN :

GET BOOK

Robust semantic scene understanding is challenging due to complex object types, as well as environmental changes caused by varying illumination and weather conditions. This thesis studies the problem of deep semantic segmentation with multimodal image inputs. Multimodal images captured from various sensory modalities provide complementary information for complete scene understanding. We provided effective solutions for fully-supervised multimodal image segmentation and few-shot semantic segmentation of the outdoor road scene. Regarding the former case, we proposed a multi-level fusion network to integrate RGB and polarimetric images. A central fusion framework was also introduced to adaptively learn the joint representations of modality-specific features and reduce model uncertainty via statistical post-processing.In the case of semi-supervised semantic scene understanding, we first proposed a novel few-shot segmentation method based on the prototypical network, which employs multiscale feature enhancement and the attention mechanism. Then we extended the RGB-centric algorithms to take advantage of supplementary depth cues. Comprehensive empirical evaluations on different benchmark datasets demonstrate that all the proposed algorithms achieve superior performance in terms of accuracy as well as demonstrating the effectiveness of complementary modalities for outdoor scene understanding for autonomous navigation.

Author : Michael Yang
Publisher : Academic Press
Page : 0 pages
File Size : 33,68 MB
Release : 2019-07-17
Category : Computers
ISBN : 9780128173589

GET BOOK

Multimodal Scene Understanding: Algorithms, Applications and Deep Learning presents recent advances in multi-modal computing, with a focus on computer vision and photogrammetry. It provides the latest algorithms and applications that involve combining multiple sources of information and describes the role and approaches of multi-sensory data and multi-modal deep learning. The book is ideal for researchers from the fields of computer vision, remote sensing, robotics, and photogrammetry, thus helping foster interdisciplinary interaction and collaboration between these realms. Researchers collecting and analyzing multi-sensory data collections - for example, KITTI benchmark (stereo+laser) - from different platforms, such as autonomous vehicles, surveillance cameras, UAVs, planes and satellites will find this book to be very useful.

Author : Marco Hutter
Publisher : Springer
Page : 701 pages
File Size : 18,38 MB
Release : 2017-11-01
Category : Technology & Engineering
ISBN : 3319673610

GET BOOK

This book contains the proceedings of the 11th FSR (Field and Service Robotics), which is the leading single-track conference on applications of robotics in challenging environments. This conference was held in Zurich, Switzerland from 12-15 September 2017. The book contains 45 full-length, peer-reviewed papers organized into a variety of topics: Control, Computer Vision, Inspection, Machine Learning, Mapping, Navigation and Planning, and Systems and Tools. The goal of the book and the conference is to report and encourage the development and experimental evaluation of field and service robots, and to generate a vibrant exchange and discussion in the community. Field robots are non-factory robots, typically mobile, that operate in complex and dynamic environments: on the ground (Earth or other planets), under the ground, underwater, in the air or in space. Service robots are those that work closely with humans to help them with their lives. The first FSR was held in Canberra, Australia, in 1997. Since that first meeting, FSR has been held roughly every two years, cycling through Asia, Americas, and Europe.

Author : Suvash Sharma
Publisher :
Page : 0 pages
File Size : 39,73 MB
Release : 2022
Category :
ISBN :

GET BOOK

Due to the mission-critical nature of the autonomous driving application, underlying algorithms for scene understanding should be given special care during their development. Mostly, they should be designed with precise consideration of accuracy and run-time. Accuracy should be considered strictly which if compromised leads to faulty interpretation of the environment that may ultimately result in accidental scenarios. On the other hand, run-time holds an important position as the delayed understanding of the scene would hamper the real-time response of the vehicle which again leads to unforeseen accidental cases. These factors come as the functions of several factors such as the design and complexity of the algorithms, nature of the encountered objects or events in the environment, weather-induced effects, etc. In this work, several novel scene understanding algorithms in terms- of semantic segmentation are devised. First, a transfer learning technique is proposed in order to transfer the knowledge from the data-rich domain to a data-scarce off-road driving domain for semantic segmentation such that the learned information is efficiently transferred from one domain to another while reducing run-time and increasing the accuracy. Second, the performance of several segmentation algorithms is assessed under the easy-to-severe rainy condition and two methods for achieving the robustness are proposed. Third, a new method of eradicating the rain from the input images is proposed. Since autonomous vehicles are rich in sensors and each of them has the capability of representing different types of information, it is worth fusing the information from all the possible sensors. Forth, a fusion mechanism with a novel algorithm that facilitates the use of local and non-local attention in a cross-modal scenario with RGB camera images and lidar-based images for road detection using semantic segmentation is executed and validated for different driving scenarios. Fifth, a conceptually new method of off-road driving trail representation, called Traversability, is introduced. To establish the correlation between a vehicle’s capability and the level of difficulty of the driving trail, a new dataset called CaT (CAVS Traversability) is introduced. This dataset is very helpful for future research in several off-road driving applications including military purposes, robotic navigation, etc.

Author : Helia Sharif
Publisher :
Page : 0 pages
File Size : 28,80 MB
Release : 2021
Category :
ISBN :

GET BOOK

“Earth is the cradle of humanity, but one cannot live in a cradle forever.” -Konstantin E. Tsiolkovsky, an early pioneer of rocketry and astronautics. Space robotics enable humans to explore beyond our home planet. Traditional techniques for tele-operated robotic guidance make it possible for a driver to direct a rover that is up to 245.55Mkm away. However, relying on manual terrestrial operators for guidance is a key limitation for exploration missions today, as real-time communication between rovers and operators is delayed by long distances and limited uplink opportunities. Moreover, autonomous guidance techniques in use today are generally limited in scope and capacity; for example, some autonomous techniques presently in use require the application of special markers on targets in order to enable detection, while other techniques provide autonomous vision-based flight navigation but only at limited altitudes in ideal visibility conditions. Improving autonomy is thus essential to expanding the scope of viable space missions. In this thesis, a fusion of monocular visible and infrared imaging cameras is employed to estimate the relative pose of a nearby target while compensating for each spectrum's shortcomings. The robustness of the algorithm was tested in a number of different scenarios by simulating harsh space environments while imaging a subject of similar characteristics to a spacecraft in orbit. It is shown that the fusion of visual odometries from two spectrums performs well where knowledge of the target's physical characteristics is limited. The result of this thesis research is an autonomous, robust vision-based tracking system designed for space applications. This appealing solution can be used onboard most spacecraft and adapted for the specific application of any given mission.

Author : Philippe Xu
Publisher :
Page : 0 pages
File Size : 33,93 MB
Release : 2014
Category :
ISBN :

GET BOOK

Image understanding is a key issue in modern robotics, computer vison and machine learning. In particular, driving scene understanding is very important in the context of advanced driver assistance systems for intelligent vehicles. In order to recognize the large number of objects that may be found on the road, several sensors and decision algorithms are necessary. To make the most of existing state-of-the-art methods, we address the issue of scene understanding from an information fusion point of view. The combination of many diverse detection modules, which may deal with distinct classes of objects and different data representations, is handled by reasoning in the image space. We consider image understanding at two levels : object detection ans semantic segmentation. The theory of belief functions is used to model and combine the outputs of these detection modules. We emphazise the need of a fusion framework flexible enough to easily include new classes, new sensors and new object detection algorithms. In this thesis, we propose a general method to model the outputs of classical machine learning techniques as belief functions. Next, we apply our framework to the combination of pedestrian detectors using the Caltech Pedestrain Detection Benchmark. The KITTI Vision Benchmark Suite is then used to validate our approach in a semantic segmentation context using multi-modal information.

Author : Pål Liljebäck
Publisher : Springer Science & Business Media
Page : 317 pages
File Size : 27,14 MB
Release : 2012-06-13
Category : Technology & Engineering
ISBN : 1447129962

GET BOOK

Snake Robots is a novel treatment of theoretical and practical topics related to snake robots: robotic mechanisms designed to move like biological snakes and able to operate in challenging environments in which human presence is either undesirable or impossible. Future applications of such robots include search and rescue, inspection and maintenance, and subsea operations. Locomotion in unstructured environments is a focus for this book. The text targets the disparate muddle of approaches to modelling, development and control of snake robots in current literature, giving a unified presentation of recent research results on snake robot locomotion to increase the reader’s basic understanding of these mechanisms and their motion dynamics and clarify the state of the art in the field. The book is a complete treatment of snake robotics, with topics ranging from mathematical modelling techniques, through mechatronic design and implementation, to control design strategies. The development of two snake robots is described and both are used to provide experimental validation of many of the theoretical results. Snake Robots is written in a clear and easily understandable manner which makes the material accessible by specialists in the field and non-experts alike. Numerous illustrative figures and images help readers to visualize the material. The book is particularly useful to new researchers taking on a topic related to snake robots because it provides an extensive overview of the snake robot literature and also represents a suitable starting point for research in this area.

Author : Richard Hartley
Publisher : Cambridge University Press
Page : 676 pages
File Size : 15,83 MB
Release : 2004-03-25
Category : Computers
ISBN : 1139449141

GET BOOK

A basic problem in computer vision is to understand the structure of a real world scene given several images of it. Techniques for solving this problem are taken from projective geometry and photogrammetry. Here, the authors cover the geometric principles and their algebraic representation in terms of camera projection matrices, the fundamental matrix and the trifocal tensor. The theory and methods of computation of these entities are discussed with real examples, as is their use in the reconstruction of scenes from multiple images. The new edition features an extended introduction covering the key ideas in the book (which itself has been updated with additional examples and appendices) and significant new results which have appeared since the first edition. Comprehensive background material is provided, so readers familiar with linear algebra and basic numerical methods can understand the projective geometry and estimation algorithms presented, and implement the algorithms directly from the book.

Author : Francisco José Perales
Publisher : Springer
Page : 141 pages
File Size : 19,46 MB
Release : 2018-07-03
Category : Computers
ISBN : 3319945440

GET BOOK

This book constitutes the refereed proceedings of the 10th International Conference on Articulated Motion and Deformable Objects, AMDO 2018, held in Palma de Mallorca, Spain, in July 2018. The 12 papers presented were carefully reviewed and selected from 26 submissions. The papers address the following topics: advanced computer graphics and immersive videogames; human modeling and animation; human motion analysis and tracking; 3D human reconstruction and recognition; multimodal user interaction and applications; ubiquitous and social computing; design tools; input technology; programming user interfaces; 3D medical deformable models and visualization; deep learning methods for computer vision and graphics; and multibiometric.