Fourier Ptychography Microscopy enables reconstructing both intensity and phase high-resolution wide-field images from multiple captures under varying illumination directions. The capture process is classically modeled using a neural network. The reconstructed object is iteratively optimized by gradient descent so the network output matches the captures. Although, this process hinges on a precise estimation of the system geometry. While previous works alternate object image refinement and LEDs positional misalignment correction, we show that geometry estimation can be efficiently integrated into the object reconstruction process, so achieving system self-calibration, and enhancing the quality of reconstructed images.

Fourier Ptychographic microscopy (FPM) which is a low-cost but among the high throughput techniques. The main aim of FPM is to increase the space bandwidth product (SBP). In a microscope, one may trade of either to have a large image with low resolution or a small image with high resolution this is commonly termed as SBP that stands for space bandwidth product and defined as a product between field of view (FoV) and bandwidth of images in frequency domain. Researchers have developed a Fourier ptychographic microscopy (FPM) by introducing hardware design and computational imaging methods that provide high resolution images and wide field for viewing the image. One of the methods is to place number of LEDs beneath the slice in microscope to accumulate the information. Therefore, this manuscript presents the fundamental concept of Ptychographic Fourier microscopy along with the state-of-the-art approaches and the analysis related to auto-calibration of LEDs. In addition to this, paper will give directions for upcoming researchers to execute the task easily.

We compare Deep Convolutional Neural Networks (DCNN) frameworks, namely AlexNet and VGGNet, for the classification of healthy and malaria-infected cells in large, grayscale, low quality and low resolution microscopic images, in the case only a small training set is available. Experimental results deliver promising results on the path to quick, automatic and precise classification in unstained images.

We compare Deep Convolutional Neural Networks (DCNN) frameworks, namely AlexNet and VGGNet, for the classification of healthy and malaria-infected cells in large, grayscale, low quality and low resolution microscopic images, in the case only a small training set is available. Experimental results deliver promising results on the path to quick, automatic and precise classification in unstrained images.

Our goal is to capture style from real human motion so it can be rendered with a virtual agent that represents this human user. We used expressivity parameters to describe motion style. As a first contribution, we propose an approach to estimate a subset of expressivity parameters defined in the literature (namely spatial extent and temporal extent) from captured motion trajectories. Second, we capture the expressivity of real users and then output it to the Greta engine that animates a virtual agent representing the user. We experimentally demonstrate that expressivity can be another clue for identifiable virtual clones of real humans.

Elongated objects have various shapes and can shift, rotate, change scale, and be rigid or deform by flexing, articulating, and vibrating, with examples as varied as a glass bottle, a robotic arm, a surgical suture, a finger pair, a tram, and a guitar string. This generally makes tracking of poses of elongated objects very challenging.We describe a unified, configurable framework for tracking the pose of elongated objects, which move in the image plane and extend over the image region. Our method strives for simplicity, versatility, and efficiency. The object is decomposed into a chained assembly of segments of multiple parts that are arranged under a hierarchy of tailored spatio-temporal constraints. In this hierarchy, segments can rescale independently while their elasticity is controlled with global orientations and local distances.While the trend in tracking is to design complex, structure-free algorithms that update object appearance online, we show that our tracker, with the novel but remarkably simple, structured organization of parts with constant appearance, reaches or improves state-of-the-art performance. Most importantly, our model can be easily configured to track exact pose of arbitrary, elongated objects in the image plane. The tracker can run up to 100 fps on a desktop PC, yet the computation time scales linearly with the number of object parts. To our knowledge, this is the first approach to generic tracking of elongated objects.

We propose a novel probabilistic tracking algorithm based on an elastic blob ensemble (EBE) which is applicable to track flexible objects. It outputs a coarse motion cue in the form of the object's location and orientation together with the location of the blobs. The main assumption is that the orientation of the whole object does not change much between neighboring frames. A discrete solution space is created in the current frame around the blobs' positions from the previous frame. Our model then promotes solutions whose orientations are close to the prior orientation, which match the modeled to the observed appearance well, and which follow modeled spatial configuration. It combines the strengths of three popular approaches to visual tracking : mean-shift tracker, particle filtering, and pictorial structures. As a result, the proposed framework tracks human upper body composed of head, torso, and hips at the rate of 40 fps on a regular desktop PC.

Haar-like features are ubiquitous in computer vision, e.g. for Viola and Jones face detection or local descriptors such as Speeded-Up-Robust-Features. They are classically computed in one pass over integral image by reading the values at the feature corners. Here we present a new, general parsing formalism for convolving them more efficiently. Our method is fully automatic and applicable to an arbitrary set of Haar-like features. The parser reduces the number of memory accesses which are the main computational bottleneck during convolution on modern computer architectures. It first splits the features into simpler kernels. Then it aligns and reuses them where applicable forming an ensemble of recursive convolution trees, which can be computed faster. This is illustrated with experiments, which show a significant speed-up over the classic approach.

Networked 3D virtual environments allow multiple users to interact over the Internet by means of avatars and to get some feeling of a virtual telepresence. However, avatar control may be tedious. 3D sensors for motion capture systems based on 3D sensors have reached the consumer market, but webcams remain more widespread and cheaper. This work aims at animating a user’s avatar by real-time motion capture using a personal computer and a plain webcam. In a classical model-based approach, we register a 3D articulated upper-body model onto video sequences and propose a number of heuristics to accelerate particle filtering while robustly tracking user motion. Describing the body pose using wrists 3D positions rather than joint angles allows efficient handling of depth ambiguities for probabilistic tracking. We demonstrate experimentally the robustness of our 3D body tracking by real-time monocular vision, even in the case of partial occlusions and motion in the depth direction.

Dans une collaboration a distance mediatisee par ordinateur, la communication entre personnes est renforcee par le contact visuel. Toutefois, la videoconference ne permet pas restituer les actions sur les objets partages. La perception mutuelle des participants et de leurs actions peut etre renforcee au moyen d'un environnement virtuel en reseau. Des representations virtuelles des utilisateurs distants (avatars) et des objets qu'ils partagent y sont rassemblees de sorte que les actions des utilisateurs sur les objets sont utilisees pour animer les avatars qui miment leurs actions. Les gestes de communication sont eux acquis par vision artificielle et restitues virtuellement en temps reel. Un systeme d'acquisition par vision artificielle des gestes et expressions du visage en 3D peut etre developpe a partir d'une unique webcam. Ce systeme a bas debit et bas cout permet d'envisager des applications grand public de telepresence virtuelle en 3D.

Our objective is to animate an embodied conversational agent (ECA) with communicative gestures rendered with the expressivity of a real human user it represents. We describe an approach to estimate a subset of expressivity parameters defined in the literature (namely spatial and temporal extent) from captured motion trajectories. We first validate this estimation against synthesis motion and then show results with real human motion. The estimated expressivity is then sent to the animation engine of an ECA that becomes a personalized autonomous representative of that user.

In a computer supported distant collaboration, communication between users can be enhanced with a visual channel. Plain videos of individual users unfortunately fail to render their joint actions on the objects they share, which limits their mutual perception. Remote interaction can be enhanced by immersing user representations (avatars) and the shared objects in a networked 3D virtual environment, so user actions are rendered by avatar mimicry. Communication gesture (not actions) are captured by real-time computer vision and rendered. We have developed a system based on a single-webcam for body and face 3D motion capture. We have used a library of communication gestures to learn statistical gesture models and used them as prior constraints for monocular motion capture, so improving tracking ambiguous poses and rendering some motion details. We have developed an open source library for real-time image analysis and computer vision that supports acceleration by consumer graphical processing units (GPUs). Finally, users are rendered with low-bandwidth avatar animation, thus opening the path to low-cost remote virtual presence at home.

Les environnements virtuels collaboratifs permettent a plusieurs utilisateurs d’interagir a distance par Internet. Ils peuvent partager une impression de telepresence en animant a distance un avatar qui les represente. Toutefois, le controle de cet avatar peut etre difficile et mal restituer les gestes de l’utilisateur. Ce travail vise a animer l’avatar a partir d’une acquisition 3D des gestes de l’utilisateur par vision monoculaire en temps reel, et a rendre la telepresence virtuelle possible au moyen d’un PC grand public equipe d’une webcam. L’approche suivie consiste a recaler un modele 3D articule de la partie superieure du corps humain sur une sequence video. Ceci est realise en cherchant iterativement la meilleure correspondance entre des primitives extraites du modele 3D d’une part et de l’image d’autre part. Le recalage en deux etapes peut proceder sur les regions, puis sur les contours. La premiere contribution de cette these est une methode de repartition des iterations de calcul qui optimise la robustesse et la precision sous la contrainte du temps-reel. La difficulte majeure pour le suivi 3D a partir d’images monoculaires provient des ambiguites 3D/2D et de l’absence d’information de profondeur. Le filtrage particulaire est desormais une approche classique pour la propagation d’hypotheses multiples entre les images. La deuxieme contribution de cette these est une amelioration du filtrage particulaire pour le recalage 3D/2D en un temps de calcul limite par des heuristiques, dont la contribution est demontee experimentalement. Un parametrage de l’attitude des bras par l’extremite de leur chaine cinematique est propose qui permet de mieux modeliser l’incertitude sur la profondeur. Enfin, l’evaluation est acceleree par calcul sur GPU. En conclusion, l’algorithme propose permet un suivi 3D robuste en temps-reel a partir d’une webcam pour une grande variete des gestes impliquant des occlusions partielles et des mouvements dans la direction de la profondeur.

A challenge for 3D motion capture by monocular vision is 3D-2D projection ambiguities that may bring incorrect poses during tracking. In this paper, we propose improving 3D motion capture by learning human gesture models from a library of gestures with variants. This library has been created with virtual human animations. Gestures are described as Gaussian Process Dynamic Models (GPDM) and are used as constraints for motion tracking. Given the raw input poses from the tracker, the gesture model helps to correct ambiguous poses. The benefit of the proposed method is demonstrated with results.

Particle filtering is known as a robust approach for motion tracking by vision, at the cost of heavy computation in a high dimensional pose space. In this work, we describe a number of heuristics that we demonstrate to jointly improve robustness and real-time for motion capture. 3D human motion capture by monocular vision without markers can be achieved in realtime by registering a 3D articulated model on a video. First, we search the high-dimensional space of 3D poses by generating new hypotheses (or particles) with equivalent 2D projection by kinematic flipping. Second, we use a semi-deterministic particle prediction based on local optimization. Third, we deterministi-cally resample the probability distribution for a more efficient selection of particles. Particles (or poses) are evaluated using a match cost function and penalized with a Gaussian probability pose distribution learned off-line. In order to achieve real-time, measurement step is parallelized on GPU using the OpenCL API. We present experimental results demonstrating robust real-time 3D motion capture with a consumer computer and webcam.

The Active Appearance Models [1] and the derived Active Models (AM) [4] allow to robustly track the face of a single user that was previously learnt, but works poorly with multiple or unknown users. Our research aims at improving the tracking robustness by learning from video databases. In this paper, we study the relation between the face texture and the parameter gradient matrix, and propose a statistical approach to dynamically fit the AM to unknown users by estimating the gradient and update matrices from the face texture. We have implemented this algorithm for real time face tracking and experimentally demonstrate its robustness when tracking multiple or unknown users' faces.

Nous nous interessons a l'acquisition 3D des gestes humains par vision monoscopique en temps reel sans marqueurs. Notre approche procede par recalage d'un modele 3D articule du corps sur une sequence video qui consiste a rechercher iterativement la position du modele et les angles d'articulation qui maximisent la correspondance entre des caracteristiques du modele 3D projete et des primitives de l'image. Nous avons precedemment decrit une mise en oeuvre a la cadence video d'un recalage initial sur les regions colorees suivi d'un recalage plus precis sur les contours [7]. Dans ce travail, nous comparons experimentalement l'erreur residuelle en fonction du temps de calcul pour chacune de ces primitives de recalage et nous proposons un compromis en fonction de la puissance de calcul disponible.

3D human motion capture by real-time monocular vision without using markers can be achieved by registering a 3D articulated model on a video. Registration consists in iteratively optimizing the match between primitives extracted from the model and the images with respect to the model position and joint angles. We extend a previous color-based registration algorithm with a more precise edge-based registration step. We present an experimental analysis of the residual error vs. the computation time and we discuss the balance between both approaches.

Virtual worlds are developing rapidly over the Internet. They are visited by avatars and staffed with Embodied Conversational Agents (ECAs). An avatar is a representation of a physical person. Each person controls one or several avatars and usually receives feedback from the virtual world on an audio-visual display. Ideally, all senses should be used to feel fully embedded in a virtual world. Sound, vision and sometimes touch are the available modalities. This paper reviews the technological developments which enable audio-visual interactions in virtual and augmented reality worlds. Emphasis is placed on speech and gesture interfaces, including talking face analysis and synthesis.