(cid:3) Image matching consists in (cid:20)nding in two images the features which represent a same feature of the observed scene(cid:17) It is a basic process of vision as soon as several images are used(cid:17) In this paper(cid:4) we propose a matching algorithm for two images(cid:4) based on the use of local geometric quasi(cid:18)invariants(cid:17) Once a (cid:20)rst match is made(cid:4) a projective approximation of the apparent motion and the epipolar geometry may be used to complete it(cid:17) In the case where we consider more than two images(cid:4) these images may be matched two by two in a (cid:20)rst step(cid:4) and a global match is deduced in a second step(cid:19) this is exposed in the last section(cid:17) The main advantages of the method presented here are the following(cid:19) it still works even if the images are noisy and the polyhedral approximation of the contours is not exact(cid:4) if the apparent motion between the images is not very small(cid:4) if the whole scene has not a single rigid motion(cid:4) if the camera is not calibrated and the camera motion between the two shots is not known(cid:0)(cid:0)(cid:0) It is thus usable in many cases where no other method is available(cid:17)

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. In Memoriam Roger Mohr Karl Tombre, Long Quan, Radu Horaud, Patrick Gros, Cordelia Schmid, Peter Sturm

Bags of visual words are the most studied image description technique in the last years. This representation of images raised new possibilities as well as new research issues. In particular, it is important to automatically determine which visual words are the most relevant to describe the images, and which ones should be ignored. This issue is a classical problem of textual information retrieval, usually addressed by the use of weighting schemes. In this paper, the most common weighting schemes from text retrieval are applied to the case of visual word-based retrieval. New weighting schemes are also proposed, and several Minkowski-like distances are tested. The experiments are performed on four different datasets that correspond to two different retrieval tasks; it allows us to bring to light some properties of visual words and weighting schemes. This study results in several findings. It first shows that the optimal setting for distances and weighting schemes depends on the nature of the visual content of the images considered. Especially, raw frequency can be the most effective weight when dealing with complex datasets; it questions the habit to systematically use the tf . idf weighting scheme. It also shows that weighting schemes and Minkowski distances have similar effect and should be used together in a consistent way. Based on these findings, general guidelines for the choice of distances and weighting schemes are proposed.

This paper addresses automatic video summarization. We propose a novel approach that relies on constraint satisfaction programming (CSP). An expert defines the general rules for summary generation. These rules are written as constraints. The (final) user can define additional constraints or enter high-level parameters of predefined constraints. This has many advantages. It clearly separates summary generation rules from the summary generation algorithm (the CSP solver here). The summary can hence be modified without reviewing the whole generation process. It also allows users to adapt the summary to the target application and to their preferences. The proposed approach is mainly dedicated to edited professional video (like TV programs). It has been extensively evaluated not only using objective measures but also through a large-scale user evaluation. This evaluation involved more than 60 people. Experiments have been performed within the challenging application of tennis match automatic summarization, using about 28.5 h of videos.

Les approches existantes pour structurer automatiquement un flux de television (i.e. reconstituer un guide de programme exact et complet), sont supervisees. Elles requierent de grandes quantites de donnees annotees manuellement, et aussi de definir a priori les types d'emissions (publicites, bandes annonces, programmes, sponsors...). Pour eviter ces deux contraintes, nous proposons une classification non supervisee. La nature multi-relationnelle de nos donnees proscrit l'utilisation des techniques de clustering habituelles reposant sur des representations sous forme attributs-valeurs. Nous proposons et validons experimentalement une technique de clustering capable de manipuler ces donnees en detournant la programmation logique inductive (PLI) pour fonctionner dans ce cadre non supervise.

Detecting audio events in Hollywood movies is a complex task due to the presence of variability between the soundtracks of the movies. This inter-movies variability is shown to impair the audio events detection results in a realistic framework. In this article, we propose to model the variability using a factor analysis technique, which we then use to compensate the audio features. The factor analysis compensation is validated using the state-of-the-art system based on multiple audio words sequences and contextual Bayesian networks which we previously developed in Penet et al. ( 2013 ). Results obtained on the same publicly available dataset for the detection of gunshots and explosions show an improvement in the handling of the variability, while keeping the robustness capabilities of the previous system. Furthermore, the system is applied to the detection of screams and proves its ability to generalise to other types of events. The obtained results also emphasise the fact that, in addition to modelling variability, adding concepts in the system may also be beneficial for the precision rates

This paper focuses on automatic video summarization. We propose a novel modeling for summary creation using constraint satisfaction programming (CSP). The proposed modeling aims to provide the summarization method with more flexibility. It allows users to easily modify the expected summary depending on their preferences or the video type. Using this new modeling, constraints become easier to formulate. Moreover, the CSP solver explores more efficiently the search space. It provides more quickly better solutions. Our model is evaluated and compared with an existing modeling on tennis videos.

This paper presents a new method to detect video repetitions in a TV stream. This method aims at reducing the number of image descriptors that have to be computed. First, shot durations are used as hash keys to propose a first set of potential repetitions. Second, a statistical test and visual descriptors are used to verify and complete these first guesses in order to improve the results by reducing the number of false detections and by retrieving the maximal length of the repeated segments. The method is tested in the context of a TV structuring application using one month of continuous TV stream.

This paper presents the work done at Technicolor and INRIA regarding the MediaEval 2013 Violent Scenes Detection task, which aims at detecting violent scenes in movies. We participated in both the objective and the subjective subtasks.

This article investigates a novel use of the well-known audio words representations to detect specific audio events, namely gunshots and explosions, in order to get more robustness towards soundtrack variability in Hollywood movies. An audio stream is processed as a sequence of stationary segments. Each segment is described by one or several audio words obtained by applying product quantization to standard features. Such a representation using multiple audio words constructed via product quantisation is one of the novelties described in this work. Based on this representation, Bayesian networks are used to exploit the contextual information in order to detect audio events. Experiments are performed on a comprehensive set of 15 movies, made publicly available. Results are comparable to the state of the art results obtained on the same dataset but show increased robustness to decision thresholds, however limiting the range of possible operating points in some conditions. Late fusion provides a solution to this issue.

The MediaEval 2012 A ect Task is the continuation of the MediaEval 2011 A ect Task and aims at detecting violence in movies. This paper presents the work done in Technicolor, INRIA and Imperial College London as a response to the task. Four different systems combining audio and visual features are presented along with a fusion of three of them. Results are reported on the test data and compared with last year's results.

In this paper, we consider the issue of structuring large TV streams. More precisely, we focus on the labeling problem: once segments have been extracted from the stream, the problem is to automatically label them according to their type (eg. programs vs. commercial breaks). In the literature, several machine learning techniques have been proposed to solve this problem: Inductive Logic Programming, numeric classifiers like SVM or decision trees... In this paper, we assimilate the problem of labeling segments to the problem of labeling a sequence of data. We propose to use a very effective approach based on another classifier: the Conditional Random Fields (CRF), a tool which has proved useful to handle sequential data in other domains. We report different experiments, conducted on some manually and automatically segmented data, with different label granularities and different features to describe segments. We demonstrate that this approach is more robust than other classification methods, in particular when it uses the neighbouring context of a segment to find its type. Moreover, we highlight that the segmentation and the choice of features to describe segments are two crucial points in the labeling process.

This paper presents BABAZ, an audio search system to search modified segments in large databases of music or video tracks. It is based on an efficient audio feature matching system which exploits the reciprocal nearest neighbors to produce a per-match similarity score. Temporal consistency is taken into account based on the audio matches, and boundary estimation allows the precise localization of the matching segments. The method is mainly intended for video retrieval based on their audio track, as typically evaluated in the copy detection task of TRECVID evaluation campaigns. The evaluation conducted on music retrieval shows that our system is comparable to a reference audio fingerprinting system for music retrieval, and significantly outperforms it on audio-based video retrieval, as shown by our experiments conducted on the dataset used in the copy detection task of TRECVID'2010 campaign.

In this paper, we propose a novel image classification framework based on patch matching. More precisely, we adapt the Hamming Embedding technique, first introduced for image search to improve the bag-of-words representation. This matching technique allows the fast comparison of descriptors based on their binary signatures, which refines the matching rule based on visual words and thereby limits the quantization error. Then, in order to allow the use of efficient and suitable linear kernel-based SVM classification, we propose a mapping method to cast the scores output by the Hamming Embedding matching technique into a proper similarity space. Comparative experiments of our proposed approach and other existing encoding methods on two challenging datasets PASCAL VOC 2007 and Caltech-256, report the interest of the proposed scheme, which outperforms all methods based on patch matching and even provide competitive results compared with the state-of-the-art coding techniques.

We investigate the use of structure learning in Bayesian networks for a complex multimodal task of action detection in soccer videos. We illustrate that classical score-oriented structure learning algorithms, such as the K2 one whose usefulness has been demonstrated on simple tasks, fail in providing a good network structure for classification tasks where many correlated observed variables are necessary to make a decision. We then compare several structure learning objective functions, which aim at finding out the structure that yields the best classification results, extending existing solutions in the literature. Experimental results on a comprehensive data set of 7 videos show that a discriminative objective function based on conditional likelihood yields the best results, while augmented approaches offer a good compromise between learning speed and classification accuracy.

This paper presents a violent shots detection system that studies several methods for introducing temporal and multimodal information in the framework. It also investigates different kinds of Bayesian network structure learning algorithms for modelling these problems. The system is trained and tested using the MediaEval 2011 Affect Task corpus, which comprises of 15 Hollywood movies. It is experimentally shown that both multimodality and temporality add interesting information into the system. Moreover, the analysis of the links between the variables of the resulting graphs yields important observations about the quality of the structure learning algorithms. Overall, our best system achieved 50% false alarms and 3% missed detection, which is among the best submissions in the MediaEval campaign.

Nous nous interessons a la structuration de flux televisuels et nous abordons plus particulierement le probleme de l'etiquetage : une fois les segments identifies, il s'agit de leur attribuer a chacun le type correspondant a son contenu semantique. Pour realiser cette tâche, differentes techniques de classification et d'apprentissage ont ete utilisees : Programmation Logique Inductive, Machines a Vecteurs de Support, Arbres de decision, etc. Dans cet article, nous proposons d'etiqueter les segments a l'aide des Champs Conditionnels Aleatoires, un outil efficace pour l'etiquetage de donnees sequentielles dans de nombreux domaines. Pour montrer l'interet de cette approche, nous presentons diverses experimentations, realisees sur des flux segmentes manuellement et automatiquement. Nous faisons egalement varier la granularite de l'etiquetage et les descripteurs caracterisant les segments. Les resultats obtenus montrent que notre approche est plus robuste que les autres methodes de classification, en particulier grâce a la prise en compte du voisinage d'un segment dans la determination de son type. De plus, nous mettons en evidence l'influence du type de segmentation et du choix des descripteurs des segments dans la qualite de l'etiquetage

In the last few decades, digital TV broadcasting has witnessed a huge interest from users against the traditional analog transmission. Many facilities have been already provided for capturing, storing, and sharing digital video content. However, navigating within TV streams is still considered as an important challenge to be faced. From a user point of view, a TV stream is represented as a sequence of programs (P) and breaks (B). While from a signal point of view, this stream is seen as a continuous flow of video and audio frames, with no external markers of the start and the end points of the included programs and no apparent structure. Most TV streams have no associated metadata to describe their structure, except the program guides produced by the TV channels. These program guides lack precision, since the TV channels cannot predict the exact duration of live programs for example. In addition, they don't provide any information about breaks like commercials or trailers. To cope with this problem, TV stream macro-segmentation or structuring has been proposed as a promising solution in the domain of video indexing. TV stream macro-segmentation consists in precisely detecting the first and the last frames of all programs and breaks (commercials, trailers, station identification, bumpers…) of a given stream, and then in annotating all these segments with some metadata. This can be performed by (1) analyzing the metadata provided with the stream (EPG, EIT…), or (2) analyzing the audio-visual stream to detect the precise start and end of programs and breaks. In this chapter we aim at providing a survey of the existing approaches in this field. Then, we discuss and compare the results of the different approaches to see how to benefit from the advantages and to overcome the limitations of each of them.