Deep neural networks virtually dominate the domain of most modern vision systems, providing high performance at a cost of increased computational complexity. Since for those systems it is often required to operate both in real-time and with minimal energy consumption (e.g., for wearable devices or autonomous vehicles, edge Internet of Things (IoT), sensor networks), various network optimisation techniques are used, e.g., quantisation, pruning, or dedicated lightweight architectures. Due to the logarithmic distribution of weights in neural network layers, a method providing high performance with significant reduction in computational precision (for 4-bit weights and less) is the Power-of-Two (PoT) quantisation (and therefore also with a logarithmic distribution). This method introduces additional possibilities of replacing the typical for neural networks Multiply and ACcumulate (MAC—performing, e.g., convolution operations) units, with more energy-efficient Bitshift and ACcumulate (BAC). In this paper, we show that a hardware neural network accelerator with PoT weights implemented on the Zynq UltraScale + MPSoC ZCU104 SoC FPGA can be at least 1.4x more energy efficient than the uniform quantisation version. To further reduce the actual power requirement by omitting part of the computation for zero weights, we also propose a new pruning method adapted to logarithmic quantisation.
Modern digital Role Playing Games are often played on some sort of a map. Creating those maps by artists can take a long time. In this paper we propose a solution for automatic creation of such maps, that can then be used as a support tool for creating a baseline map to work on by artists, or as an automatic map generator. The general approach of creating maps using our solution is briefly explained, and then a closer look at the ground part of the map creation process is taken. Our algorithm takes as an input a list of tiles to be used and a rough sketch of a map to be generated, prepared with specific colours. It then creates a tile grid from that sketch using simple downscaling. That tile grid is then corrected using methods proposed in this paper, so that it does not have sharp edges and mismatched tiles. Finally, using tiles from input, the grid is transformed into an output map that closely resembles the sketch in its structure. Our solution is tested in practice by means of several exemplary sketches, each accompanied with comment explaining the significance of obtained result.
3D object detection from LiDAR sensor data is an important topic in the context of autonomous cars and drones. In this paper, we present the results of experiments on the impact of backbone selection of a deep convolutional neural network on detection accuracy and computation speed. We chose the PointPillars network, which is characterised by a simple architecture, high speed, and modularity that allows for easy expansion. During the experiments, we paid particular attention to the change in detection efficiency (measured by the mAP metric) and the total number of multiply-addition operations needed to process one point cloud. We tested 10 different convolutional neural network architectures that are widely used in image-based detection problems. For a backbone like MobilenetV1, we obtained an almost 4x speedup at the cost of a 1.13% decrease in mAP. On the other hand, for CSPDarknet we got an acceleration of more than 1.5x at an increase in mAP of 0.33%. We have thus demonstrated that it is possible to significantly speed up a 3D object detector in LiDAR point clouds with a small decrease in detection efficiency. This result can be used when PointPillars or similar algorithms are implemented in embedded systems, including SoC FPGAs. The code is available at https://github.com/vision-agh/pointpillars\_backbone.
A method of human skeleton-tracking and -refinement, and feature extraction for two-person interaction recognition in video is proposed. Its purpose is to properly reassign the same person-representing skeletons, approximate the missing joints and extract meaningful relational features. In addition, based on the created feature streams, two different multi-stream deep neural networks are designed to perform data transformation and interaction classification. They provide different relations between model complexity and performance quality. The first one is an ensemble of "weak" pose-based action classifiers, which are trained on different time-phases of an interaction. At the same time, the overall classification result is a time-driven aggregation of weighted combinations of their results. In the second approach, three input feature streams were created, which fed a triple-stream LSTM network. Both network models were trained and tested on the interaction subset of the NTU RGB+D data set, showing comparable performance with the best reported CNN- and Graphic CNN-based classifiers.
In recent years Human Object Interaction (HOI) detection has experienced rapid performance growth mainly due to the development of various deep learning-based methods and algorithms. One of the most popular approaches to this task is three-stream architecture consisting of three processing paths: human-stream, object-stream, and relation-stream, gathering information on the scene using features extracted from particular image elements. The total amount of features depends on the detailed processing schemes and model parameters. The increase in the number of features does not necessarily increase the final efficiency of the HOI detection method. This paper focuses on the relation between the quantity of features used to detect HOI and its predictive efficiency. In our study, we investigate the influence of the number of features on the final detection results. Several experiments were conducted to validate various model configurations of different features considered.
