Object Pose Estimation is a crucial component in robotic grasping and augmented reality. Learning based approaches typically require training data from a highly accurate CAD model or labeled training data acquired using a complex setup. We address this by learning to estimate pose from weakly labeled data without a known CAD model. We propose to use a NeRF to learn object shape implicitly which is later used to learn view-invariant features in conjunction with CNN using a contrastive loss. While NeRF helps in learning features that are view-consistent, CNN ensures that the learned features respect symmetry. During inference, CNN is used to predict view-invariant features which can be used to establish correspondences with the implicit 3d model in NeRF. The correspondences are then used to estimate the pose in the reference frame of NeRF. Our approach can also handle symmetric objects unlike other approaches using a similar training setup. Specifically, we learn viewpoint invariant, discriminative features using NeRF which are later used for pose estimation. We evaluated our approach on LM, LM-Occlusion, and T-Less dataset and achieved benchmark accuracy despite using weakly labeled data.

Even if the possibilities in the industrial metaverse still sound like dreams of the future, the solutions that can already be implemented today are impressive. This article focuses on applications and use cases in Augmented Reality (AR), an essential building block of the industrial metaverse. For this purpose, information available in AR are presented, which are used in several use cases. Two exemplary proof-of-concepts illustrate what is already possible today.

Auch wenn die Möglichkeiten im Industrial Metaverse noch nach Zukunftsmusik klingen, beeindrucken die heute schon realisierbaren Lösungen. Dieser Beitrag fokussiert sich auf Anwendungen und Use Cases in Augmented Reality (AR), einem wesentlichen Baustein des Industrial Metaverse. Dazu werden in AR verfügbare Informationen vorgestellt, die in mehreren Use Cases genutzt werden. Zwei beispielhafte Proof-of-Concepts veranschaulichen was bereits heute möglich ist.

Object pose estimation is a crucial task in computer vision and augmented reality. One of its key challenges is the difficulty of annotation of real training data and the lack of textured CAD models. Therefore, pipelines which do not require CAD models and which can be trained with few labeled images are desirable. We propose a weakly-supervised approach for object pose estimation from RGB-D data using training sets composed of very few labeled images with pose annotations along with weakly-labeled images with ground truth segmentation masks without pose labels. We achieve this by learning to annotate weakly-labeled training data through shape alignment while simultaneously training a pose prediction network. Point cloud alignment is performed using structure and rotation-invariant feature-based losses. We further learn an implicit shape representation, which allows the method to work without the known CAD model and also contributes to pose alignment and pose refinement during training on weakly labeled images. The experimental evaluation shows that our method achieves state-of-the-art results on LineMOD, Occlusion-LineMOD and TLess despite being trained using relative poses and on only a fraction of labeled data used by the other methods. We also achieve comparable results to state-of-the-art RGB-D based pose estimation approaches even when further reducing the amount of unlabeled training data. In addition, our method works even if relative camera poses are given instead of object pose annotations which are typically easier to obtain.

While convolutional neural networks are dominating the field of computer vision, one usually does not have access to the large amount of domain-relevant data needed for their training. Therefore, it has become common practice to use available synthetic samples along domain adaptation schemes to prepare algorithms for the target domain. Tackling this problem from a different angle, we introduce a pipeline to map unseen target samples into the synthetic domain used to train task-specific methods. Denoising the data and retaining only the features these recognition algorithms are familiar with, our solution greatly improves their performance. As this mapping is easier to learn than the opposite one (i.e., to generate realistic features to augment the source samples), we demonstrate how our whole solution can be trained purely on augmented synthetic data and still performs better than methods trained with domain-relevant information (e.g., real images or realistic textures for the 3D models). Applying our approach to object recognition from texture-less CAD data, we present a custom generative network which fully utilizes the purely geometrical information to learn robust features and to achieve a more refined mapping for unseen color images.

With the increasing success of deep learning in various applications, there is an increasing need to have deep models that can be used for deployment in real-time and/or resource constrained scenarios. In this context, this paper analyzes the pruning of deep models for object detection in order to reduce the number of weights and hence the number of computations. Very deep networks based on ResNet like architectures, like YOLOv3 have unique challenges when attempting to prune them. This paper proposes a network pruning technique based on agglomerative clustering for the feature extractor and using mutual information for the detector. The performance of the proposed techniques is also compared with that of a relatively shallow network, i.e., YOLOv2. A compression percentage of around 30% results in a 10% drop of mean average precision (mAP) in YOLOv3, whereas in YOLOv2 the drop was around 6% on the COCO dataset.

We present a method to incrementally generate complete 2D or 3D scenes with the following properties: (a) it is globally consistent at each step according to a learned scene prior, (b) real observations of a scene can be incorporated while observing global consistency, (c) unobserved regions can be hallucinated locally in consistence with previous observations, hallucinations and global priors, and (d) hallucinations are statistical in nature, i.e., different scenes can be generated from the same observations. To achieve this, we model the virtual scene, where an active agent at each step can either perceive an observed part of the scene or generate a local hallucination. The latter can be interpreted as the agent's expectation at this step through the scene and can be applied to autonomous navigation. In the limit of observing real data at each point, our method converges to solving the SLAM problem. It can otherwise sample entirely imagined scenes from prior distributions. Besides autonomous agents, applications include problems where large data is required for building robust real-world applications, but few samples are available. We demonstrate efficacy on various 2D as well as 3D data.

Applications like autonomous driving, surveillance, or any application that demands scene analysis requires object detection, semantic segmentation and instance segmentation. In this paper, we focus on the problem of detecting each instance of a specific category of objects, specifically persons. A novel method for object detection is proposed based on a deep counting model. The feature extractor of the deep counting model is extended with additional layers for segmenting specific instances. While the feature extractor of the deep counting model already focuses on the persons in the scene, the segmentation layers help to get a more accurate estimation of the foreground with persons and the instance segmentation is able to estimate separate instances of persons. Our proposed method outperforms other methods on the CUHK08 dataset with an Average Miss Rate (AMR) of 14% and on the PETS09 dataset with an AMR of 41%.

With the increasing availability of large databases of 3D CAD models, methods for depth-based recognition of localized objects can be trained on an uncountable number of synthetically rendered images. However, discrepancies with the real data acquired from various depth sensors still noticeably impede progress. Previous works adopted unsupervised approaches to generate more realistic depth data, but they all require real scans for training, even if unlabeled. This still represents a strong requirement, especially when considering real-life/industrial settings where real training images are hard or impossible to acquire, but texture-less 3D models are available. We thus propose a novel approach leveraging only CAD models to bridge the realism gap. Purely trained on synthetic data, playing against an extensive augmentation pipeline in an unsupervised manner, our generative adversarial network learns to effectively segment depth images and recover the clean synthetic-looking depth information even from partial occlusions. As our solution is not only fully decoupled from the real domains but also from the task-specific analytics, the pre-processed scans can be handed to any kind and number of recognition methods also trained on synthetic data. Through various experiments, we demonstrate how this simplifies their training and consistently enhances their performance, with results on par with the same methods trained on real data, and better than usual approaches doing the reverse mapping.

Deep convolutional neural networks (CNNs) have achieved tremendous success in image recognition tasks. However, the performance of CNNs degrade in situations where the input image is degraded by compression artifacts, blur or noise. In this paper, we analyze some of the common CNNs for degradations in images caused by Gaussian noise, blur as well as compression using JPEG and JPEG 2000 for the full range of quality factors. Moreover, we propose a method to improve the performance of CNNs for image classification in the presence of input images with degradations based on a master-slave architecture. Our method was found to perform well for individual and combined degradations.

Pedestrian detection is an important task in surveillance applications and becomes particularly challenging when pedestrians are close together or occluding one another. This paper presents a novel approach to detect pedestrians in such challenging scenarios. A deep convolutional neural network trained for counting is specialized to count one pedestrian. The feature extractor learned thereby is exploited to detect one pedestrian at a time iteratively. For the base counting model and the specialization, extensive annotation efforts are not required since only a single number at the image level is used. Use of our method on pedestrian datasets with occlusion showed an improvement in the average miss rate values as compared to other methods for handling occlusion.

Pedestrian detection is an important task for applications like surveillance, driver assistance systems and autonomous driving. We present a novel approach for detecting pedestrians using a deep convolutional neural network (CNN) trained for counting pedestrians. Our method avoids the need for annotation of the position of the pedestrians in the training data via bounding boxes. The deconvolved outputs of the filters of the trained counting model are used to detect the pedestrians. The average miss rate values on the tested datasets were found to be in the same range as other methods in spite of a simpler training using only pedestrian counts. This method is found to be suitable for detecting pedestrians in crowded scenes with occlusion as well as less crowded scenes.

Counting pedestrians in surveillance applications is a com m n scenario. However, it is often challenging to obtain sufficient annotated training data, especially so fo r creating models using deep learning which require a large amount of training data. To address this problem, thi s paper explores the possibility of training a deep convolutional neural network (CNN) entirely from syntheti cally generated images for the purpose of counting pedestrians. Nuances of transfer learning are exploited to train models from a base model trained for image classification. A direct approach and a hierarchical approa ch are used during training to enhance the capability of the model for counting higher number of pedestrians. The t rained models are then tested on natural images of completely different scenes captured by different acqui sition systems not experienced by the model during training. Furthermore, the effectiveness of the cross entr opy cost function and the squared error cost function are evaluated and analyzed for the scenario where a model is t rained entirely using synthetic images. The performance of the trained model for the test images from the target site can be improved by fine-tuning using the image of the background of the target site.

In this paper, we address the problem of 3D object instance recognition and pose estimation of localized objects in cluttered environments using convolutional neural networks. Inspired by the descriptor learning approach of Wohlhart et al., we propose a method that introduces the dynamic margin in the manifold learning triplet loss function. Such a loss function is designed to map images of different objects under different poses to a lower-dimensional, similarity-preserving descriptor space on which efficient nearest neighbor search algorithms can be applied. Introducing the dynamic margin allows for faster training times and better accuracy of the resulting low-dimensional manifolds. Furthermore, we contribute the following: adding in-plane rotations (ignored by the baseline method) to the training, proposing new background noise types that help to better mimic realistic scenarios and improve accuracy with respect to clutter, adding surface normals as another powerful image modality representing an object surface leading to better performance than merely depth, and finally implementing an efficient online batch generation that allows for better variability during the training phase. We perform an exhaustive evaluation to demonstrate the effects of our contributions. Additionally, we assess the performance of the algorithm on the large BigBIRD dataset to demonstrate good scalability properties of the pipeline with respect to the number of models.

L'invention concerne un systeme de transmission de donnees video d'un serveur a un client. Le systeme comprend une premiere unite de codage concue pour transmettre des donnees video avec une premiere qualite du serveur au client sous la forme d'une diffusion en direct, et une seconde unite de codage qui est concue pour memoriser les donnees video avec une seconde qualite dans une unite memoire (13) et pour transmettre, en reponse a un signal de demande provenant du client (2), les donnees video codees avec la seconde qualite de l'unite memoire (13) au client (2), la seconde qualite etant superieure a la premiere qualite. Le systeme selon l'invention permet de transmettre des donnees video d'un environnement medical, par exemple d'une salle d'operation, a un expert externe par le biais d'un reseau. Les donnees video se presentent sous la forme d'une diffusion en direct en basse qualite et peuvent en outre etre fournies en haute qualite sur demande des experts. L'invention concerne egalement un procede de transmission de donnees video d'un serveur a un client.

Pixelwise linear prediction using backward-adaptive least-squares or weighted least-squares estimation of prediction coefficients is currently among the state-of-the-art methods for lossless image compression. While current research is focused on mean intensity prediction of the pixel to be transmitted, best compression requires occurrence probability estimates for all possible intensity values. Apart from common heuristic approaches, we show how prediction error variance estimates can be derived from the (weighted) least-squares training region and how a complete probability distribution can be built based on an autoregressive image model. The analysis of image stationarity properties further allows deriving a novel formula for weight computation in weighted least-squares proofing and generalizing ad hoc equations from the literature. For sparse intensity distributions in non-natural images, a modified image model is presented. Evaluations were done in the newly developed C++ framework volumetric, artificial, and natural image lossless coder (Vanilc), which can compress a wide range of images, including 16-bit medical 3D volumes or multichannel data. A comparison with several of the best available lossless image codecs proofs that the method can achieve very competitive compression ratios. In terms of reproducible research, the source code of Vanilc has been made public.

This paper presents a moving object detection algorithm for H.264/AVC video streams that is applied in the compressed domain. The method is able to extract and analyze several syntax elements from any H.264/AVC-compliant bit stream. The number of analyzed syntax elements depends on the mode in which the method operates. The algorithm is able to perform either a spatiotemporal analysis in a single step or a two-step analysis that starts with a spatial analysis of each frame, followed by a temporal analysis of several subsequent frames. Thereby, in each mode either only (sub-)macroblock types and partition modes or, additionally, quantization parameters are analyzed. The evaluation of these syntax elements enables the algorithm to determine a “weight” for each 4×4 block of pixels that indicates the level of motion within this block. A final segmentation after creating these weights segments each frame to foreground and background and hence indicates the positions and sizes of all moving objects. Our experiments show that the algorithm is able to efficiently detect moving objects in the compressed domain and that it is configurable to process a large number of parallel bit streams in real time.