The Cherenkov Telescope Array (CTA) is the future ground-based gamma-ray observatory and will be composed of two arrays of imaging atmospheric Cherenkov telescopes (IACTs) located in the Northern and Southern hemispheres respectively. The first CTA prototype telescope built on-site, the Large-Sized Telescope (LST-1), is under commissioning in La Palma and has already taken data on numerous known sources. IACTs detect the faint flash of Cherenkov light indirectly produced after a very energetic gamma-ray photon has interacted with the atmosphere and generated an atmospheric shower. Reconstruction of the characteristics of the primary photons is usually done using a parameterization up to the third order of the light distribution of the images. In order to go beyond this classical method, new approaches are being developed using state-of-the-art methods based on convolutional neural networks (CNN) to reconstruct the properties of each event (incoming direction, energy and particle type) directly from the telescope images. While promising, these methods are notoriously difficult to apply to real data due to differences (such as different levels of night sky background) between Monte Carlo (MC) data used to train the network and real data. The GammaLearn project, based on these CNN approaches, has already shown an increase in sensitivity on MC simulations for LST-1 as well as a lower energy threshold. This work applies the GammaLearn network to real data acquired by LST-1 and compares the results to the classical approach that uses random forests trained on extracted image parameters. The improvements on the background rejection, event direction, and energy reconstruction are discussed in this contribution.

Imaging atmospheric Cherenkov telescopes (IACT) data require an important analysis in order to reconstruct events and obtain a photon list.The state-of-the-art reconstruction is made of several steps including image analysis, features extraction and machine learning.Since the 2012 Ima-geNet breakthrough, deep learning advances have shown dramatic improvements in data analysis across a variety of fields.Convolutional neural networks look particularly suited to the task of analysing IACT camera images for event reconstruction as they provide a way to reconstruct photon list directly from raw images, skipping the pre-processing steps.Moreover, despite demanding important computing resources to be trained and optimised, neural networks show very good performances during execution, making them viable for real-time analysis for the future generation of IACT.Here we present GammaLearn , a python framework providing the tools and environment to easily train neural networks on IACT data.Relying on PyTorch, it allows the use of indexed convolution on images with non-cartesian pixel lattices predominant in IACT for the low-level operations and offers a simple configuration file-based workflow, producing the trained model, training estimators as well as higher level results.The proposed framework is modular and straightforward to customize by end users.It has been tested and validated on the analysis of the Cherenkov Telescope Array simulated data.

Gamma-ray reconstruction from Cherenkov telescope data is multi-task by nature in astrophysics. The image recorded in the Cherenkov camera pixels relates to the type, energy, incoming direction and distance of a particle from a telescope observation. We propose γ-PhysNet, a physically inspired multi-task deep neural network for gamma/proton particle classification, and gamma energy and direction reconstruction. We compare its performance with single task networks on Monte Carlo simulated data and demonstrate the interest of reconstructing the impact point as an auxiliary task. We also show that γ-PhysNet outperforms a widespread analysis method for gamma-ray reconstruction. Finally, we study attention methods to solve relevant use cases. All the experiments are conducted in the context of single telescope analysis for the Cherenkov Telescope Array data

The Cherenkov Telescope Array is the future of ground-based gamma-ray astronomy. Its first prototype telescope built on-site, the Large Size Telescope 1, is currently under commissioning and taking its first scientific data. In this paper, we present for the first time the development of a full-event reconstruction based on deep convolutional neural networks and its application to real data. We show that it outperforms the standard analysis, both on simulated and on real data, thus validating the deep approach for the CTA data analysis. This work also illustrates the difficulty of moving from simulated data to actual data.

In the astrophysics domain, the detection and description of gamma rays is a research direction for our understanding of the universe. Gamma-ray reconstruction from Cherenkov telescope data is multi-task by nature. The image recorded in the Cherenkov camera pixels relates to the type, energy, incoming direction and distance of a particle from a telescope observation. We propose γ -PhysNet, a physically inspired multi-task deep neural network for gamma/proton particle classification, and gamma energy and direction reconstruction. As ground truth does not exist for real data, γ -PhysNet is trained and evaluated on large-scale Monte Carlo simulations. Robustness is then crucial for the transfer of the performance to real data. Relying on a visual explanation method, we evaluate the influence of attention on the variability due to weight initialization, and how it helps improve the robustness of the model. All the experiments are conducted in the context of single telescope analysis for the Cherenkov Telescope Array simulated data analysis.

The Cherenkov Telescope Array (CTA) is the next generation ground-based observatory for γ-ray astronomy. It will be used to study γ-ray sources, allowing to better understand the Universe. One order of magnitude more sensitive than the current generation of experiments, CTA will propose unseen challenges to standard reconstruction methods. The GammaLearn project offers to apply deep learning as a part of the analysis of CTA data. Its goal is to separate the γ photons from cosmic particles, and reconstruct the γ photon parameters (energy and arrival direction) from noisy unconventional images, with expected better performance and faster reconstruction than standard methods. Here we present a complete reconstruction of IACT events using state-of-the-art deep learning techniques. The network is then applied in the single telescope context of the LST1, the first CTA telescope prototype built on the Northern hemisphere site (La Palma, Canary Island). We show that the full event reconstruction is possible with a single multi-task network, reducing the computing needs.

•Image data conditioning issue: the paper first highlights that referring 2D spatial convolution to its 1D Hilbert based instance is highly accurate for information compressibility upon image frames associated with a wide class of video files.•Video library conditioning issue: because of the above compressibility, the paper proposes converting 2D + X data volume into a single meta-image file format called timed-image, prior to machine learning frameworks. This conversion is such that any 2D frame of the 2D + X data is reshaped as a 1D array indexed by a Hilbert space-filling curve and the third variable X of the initial file format becomes the second variable in the meta-image format.•Sensitive action recognition benchmark: the paper provides two datasets having respectively 2 and 3 violence video categories. The datasets involve visual non-violent, moderate and extreme violence actions.•Sensitive action recognition issue: outstanding 2-level and 3-level violence classification results are obtained from a deep convolutional neural networks trained from scratch and operating on meta-image databases.

In recent years, semantic segmentation has become one of the most active tasks of the computer vision field. Its goal is to group image pixels into semantically meaningful regions. Deep learning methods, in particular those who use convolutional neural network (CNN), have shown a big success for the semantic segmentation task. In this paper, we will introduce a semantic segmentation system using a reinforced fully convolutional densenet with multiscale kernel prediction method. Our main contribution is to build an encoder-decoder based architecture where we increase the width of dense block in the encoder part by conducting recurrent connections inside the dense block. The resulting network structure is called wider dense block where each dense block takes not only the output of the previous layer but also the initial input of the dense block. These recurrent structure emulates the human brain system and helps to strengthen the extraction of the target features. As a result, our network becomes deeper and wider with no additional parameters used because of weights sharing. Moreover, a multiscale convolutional layer has been conducted after the last dense block of the decoder part to perform model averaging over different spatial scales and to provide a more flexible method. This proposed method has been evaluated on two semantic segmentation benchmarks: CamVid and Cityscapes. Our method outperforms many recent works from the state of the art.

Smartphones bring a new way to scan and digitalize written documents by taking pictures. This enables new document analysis applications to emerge. As a counterpart, unsupervised document capturing brings new challenges mainly related to target document localization and high quality text recognition. In this context, this work addresses automatic sale receipt understanding in an industrial context. It relies on the extraction of accurate and essential consumption data even with low quality receipt captures. We propose a tool chain that combines Deep Neural Networks and traditional image processing to ensure accurate automatic data extraction. The proposed workflow is evaluated globally by the analysis of the quality of the text recognition at the end of the processing.

The Cherenkov Telescope Array (CTA) is the next generation of ground-based gamma-ray telescopes for gamma-ray astronomy. Two arrays will be deployed composed of 19 telescopes in the Northern hemisphere and 99 telescopes in the Southern hemisphere. Due to its very high sensitivity, CTA will record a colossal amount of data that represent a computing challenge to the reconstruction software. Moreover, the vast majority of triggered events come from protons that represent a background for gamma-ray astronomy. Deep learning developments in the last few years have shown tremendous improvements in the analysis of data in many domains. Thanks to the huge amount of simulated data and later of real data, produced by CTA, these algorithms look well-suited and very promising. Moreover, the trained neural networks show very good computing performances during execution. Here we present a first study of deep learning architectures applied to CTA simulated data to perform the reconstruction of the particles energy and incoming direction and the development of a specific framework, GammaLearn, to accomplish this task.

The present paper introduces convolution and pooling operators for indexed images. These operators can be used on images that do not provide Cartesian grids of pixels, as long as a list of neighbor’s indices can be provided for each pixel. They are foreseen being useful for convolutional neural networks (CNN) applied to special sensors, especially in science, without requiring image pre-processing. The present work explains the method and its implementation in the Pytorch framework and shows an application of the indexed kernels to the classification task of images with hexagonal lattices using CNN. The obtained results show that the method gives the same performances as the standard convolution kernels. Indexed convolution thus makes deep neural network frameworks more general and capable of addressing unconventional image lattices. The current implementation, as well as code to reproduce the experiments described in this paper are made available as open-source resources on the repository www.github.com/IndexedConv.

In the Remote Sensing (RS) field, the main purpose behind the use of classical deep learning approaches is to discover and learn the content of a variety of datasets now freely available (Landsat, Sentinel...). Basically, the most performing deep learning approaches used in this context have been deployed in order to map high-dimensional rich images into class labels or execute per pixel supervised segmentation to obtain semantic maps such as the one detailed in [2] and [3]. Although such architectures have witnessed a striking success, they still are limited in terms of fulfilling non-supervised clustering tasks. However, the serious lack of richly annotated RS datasets combined with the important time dependent variability of its spatial information deflects the interest toward the use of non supervised approaches. Such challenge is emphasized when using the RS data annotations for the analysis of temporal changes. It goes without saying that the introduction of auto-encoders as an unsupervised learning method, has allowed the extraction of accurate representations of the data via latent codes (the layer about which the network is symmetric) and back-propagation techniques. In fact, Auto-encoders are in general built based on two components, an encoder and a decoder. The encoder will read the input and compress it to a compact representation while the decoder will use this representation to try recreating the input. The goal behind the use of such process in Remote Sensing tasks is to learn interpretable data representations with the minimum reconstruction error possible that can be later on used to perform classical clustering methods such the k-means and be interpreted by human experts. Vanilla Auto-encoders are however hard to train. Recently, Variational Auto-encoders [4] allow us to perform both learning and generating data samples using the latent variables Space. However, their generated samples tend to be slightly blurry and the learned representations are quite in-accurate.

Recently, a variety of approaches have been enriching the field of remote sensing (RS) image processing and analysis. Unfortunately, existing methods remain limited to the rich spatiospectral content of today's large data sets. It would seem intriguing to resort to deep learning (DL)-based approaches at this stage with regard to their ability to offer accurate semantic interpretation of the data. However, the specificity introduced by the coexistence of spectral and spatial content in the RS data sets widens the scope of the challenges presented to adapt DL methods to these contexts. Therefore, the aim of this paper is first to explore the performance of DL architectures for the RS hyperspectral data set classification and second to introduce a new 3-D DL approach that enables a joint spectral and spatial information process. A set of 3-D schemes is proposed and evaluated. Experimental results based on well-known hyperspectral data sets demonstrate that the proposed method is able to achieve a better classification rate than state-of-the-art methods with lower computational costs.

In the computer vision field, semantic segmentation represents a very interesting task. Convolutional Neural Network methods have shown their great performances in comparison with other semantic segmentation methods. In this paper, we propose a multiscale fully convolutional DenseNet approach for semantic segmentation. Our approach is based on the successful fully convolutional DenseNet method. It is reinforced by integrating a multiscale kernel prediction after the last dense block which performs model averaging over different spatial scales and provides more flexibility of our network to presume more information. Experiments on two semantic segmentation benchmarks: CamVid and Cityscapes have shown the effectiveness of our approach which has outperformed many recent works.

As a general rule, data analytics are now mandatory for companies. Scanned document analysis brings additional challenges introduced by paper damages and scanning quality. In an industrial context, this work focuses on the automatic understanding of sale receipts which enable access to essential and accurate consumption statistics. Given an image acquired with a smart-phone, the proposed work mainly focuses on the first steps of the full tool chain which aims at providing essential information such as the store brand, purchased products and related prices with the highest possible confidence. To get this high confidence level, even if scanning is not perfectly controlled, we propose a double check processing tool-chain using Deep Convolutional Neural Networks (DCNNs) on one hand and more classical image and text processings on another hand. The originality of this work relates in this double check processing and in the joint use of DCNNs for different applications and text analysis.

We are interested in the restoration of blurred colour images corrupted by additive noise. We present a new model for colour image enhancement based on coupling diffusion to shock filter without creating colour artefacts. The suggested model is based on using single vectors of the gradient magnitude and the second derivatives in order to relate different colour components of the image.

With the rapid development of Remote Sensing acquisition techniques, there is a need to scale and improve processing tools to cope with the observed increase of both data volume and richness. Among popular techniques in remote sensing, Deep Learning gains increasing interest but depends on the quality of the training data. Therefore, this paper presents recent Deep Learning approaches for fine or coarse land cover semantic segmentation estimation. Various 2D architectures are tested and a new 3D model is introduced in order to jointly process the spatial and spectral dimensions of the data. Such a set of networks enables the comparison of the different spectral fusion schemes. Besides, we also assess the use of a " noisy ground truth " (i.e. outdated and low spatial resolution labels) for training and testing the networks.