With the development of nanotechnology, it became possible to design and assemble nano-objects. For robust and reliable automation processes, handling and manipulation tasks at the nanoscale is increasingly required over the last decade. Vision is one of the most indispensable ways to observe the world in micrioscale and nanoscale. Vision-based control is an efficient solution for control problems in robotics. In this thesis, we address the issue of micro- and nano-positioning by visual servoing in a Scanning Electron Microscope (SEM). As the fundamental knowledge, the SEM image formation and SEM vision geometry models are studied at first. A nonlinear optimization process for SEM calibration has been presented considering both perspective and parallel projection model. In this study, it is found that it is difficult to observe the depth information from the variation of the pixel position of the sample in SEM image at high magnification. In order to solve the problem that the motion along the depth direction is not observable in a SEM, the image defocus information is considered as a visual feature to control the motion along the depth direction. A hybrid visual servoing scheme has been proposed for 6-DoF micro-positioning task using both image defocus information and image photometric information. It has been validated using a parallel robot in a SEM. Based on the similar idea, a closed-loop control scheme for SEM autofocusing task has been introduced and validated by experiments. In order to achieve the visual guidance in a SEM, a template-based visual tracking and 3D pose estimation framework has been proposed. This method is robust to the defocus blur caused by the motion along the depth direction since the defocus level is modeled in the visual tracking framework.

Visual tracking and estimation of the 3D posture of a micro/nano-object is a key issue in the development of automated manipulation tasks using the visual feedback. The 3D posture of the micro-object is estimated based on a template matching algorithm. Nevertheless, a key challenge for visual tracking in a scanning electron microscope (SEM) is the difficulty to observe the motion along the depth direction. In this paper, we propose a template-based hybrid visual tracking scheme that uses luminance information to estimate the object displacement on x-y plane and uses defocus information to estimate object depth. This approach is experimentally validated on 4-DoF motion of a sample in a SEM.

In this paper, a novel approach of SEM calibration based on non-linear minimization process is presented. The SEM calibration for the intrinsic parameters are achieved by an iterative non-linear optimization algorithm which minimize the registration error between the current estimated position of the pattern and its observed position. The calibration can be achieved by one image and multiple images of calibration pattern. Perspective and parallel projection models are addressed in this approach. The experimental results show the efficiency and accuracy of the proposed method.

In this paper, we present a full scale autofocus approach for scanning electron microscope (SEM). The optimal focus (in-focus) position of the microscope is achieved by maximizing the image sharpness using a vision-based closed-loop control scheme. An iterative optimization algorithm has been designed using the sharpness score derived from image gradient information. The proposed method has been implemented and validated using a tungsten gun SEM at various experimental conditions like varying raster scan speed, magnification at real-time. We demonstrate that the proposed autofocus technique is accurate, robust and fast.

Direct photometric visual servoing uses only the pure image information as a visual feature, instead of using classic geometric features such as points or lines. It was demonstrated efficiently in 6 degrees of freedom (DoF) positioning. However, in micro-scale, using only image intensity as a visual feature performs unsatisfactorily in cases where the photometric variation is low, such as motions along vision sensor's focal axis under a high magnification. In order to improve the performance and accuracy in those cases, an approach using hybrid visual features is proposed in this paper. Image gradient is employed as a visual feature on z axis while image intensity is used on the other 5 DoFs to control the motion. A 6-DoF micro-positioning task is accomplished by this hybrid visual servoing scheme. The experimental results obtained on a parallel positioning micro-stage under a digital microscope show the robustness and efficiency of the proposed method.

In this paper, an approach for 6-DoF automatic micropositioning is presented. It involves a closed-loop visual servoing scheme in order to achieve eye-to-hand positioning task in micro-scale. Instead of using classical visual features in the servoing scheme, pure image photometric information from the vision sensor is employed to compute the control law for micropositioning. The approach is validated in simulation as well as experimentally on a parallel positioning stage and a digital microscope at low magnification. Experimental results show the accuracy and efficiency of this control scheme.

Aiming at the limitation of correlation method of subjective assessment and its poor effect when facing the problems of local optimum,nonlinearity,higher dimension and small sample,the objective assessment indexes of image quality such as Mean Square Error(MSE),Peak Signal to Noise Ratio(PSNR) and Singular Value Decomposition(SVD),and the scores from LIVE database are used as the learning sample.By using Support Vector Machine(SVM) to train a correlation function between subjective and objective and Genetic Algorithm(GA) for parameter optimization,a perfect model is obtained to achieve the consistency of subjective and objective.Experimental results show that compared with traditional methods,the assessment with the method is more accurate.

Structural similarity (SSIM) is one image quality assessment metric that focuses on the statistic information in the spatial domain. It cannot reflect the small details of the contrast and the changing of texture, which can be perceived by human visual system, because SSIM cannot detect the distortion image with aliasing and blur effectively. This paper proposes a new image quality assessment metric called structural similarity based on global phase coherence (GPC-SSIM), which considers both the structural information in the spatial domain and the phase characteristics in the frequency domain. Through experiments, as the level of blur and aliasing of an image gets more and more serious, the dynamic range of the results obtained through SSIM is 0.6~1, while the ones through the new assessment index GPC_SSIM is 0~1. Thus GPC-SSIM is more sensitive to the blur and aliasing of image and can give more accurate assessment results for various kinds of degraded images than SSIM.

In this paper, we propose a correlation method to assess image quality based on support vector machine (SVM) and genetic algorithm (GA). Instead of the simple linear function to correlate objective indicators with subjective scores of images, we introduce SVM for the correlation function, make GA as the search algorithm, and finally get the image quality assessment model. The results of experiments show: It is effective to introduce SVM to make correlation between objective indicators and subjective scores for image quality assessment; the correlation between objective indicators and subjective scores is better by using SVM based on GA.

In this paper, a novel approach of defocus blur image restoration is proposed, which is based on Global Phase Coherence (GPC). As a good sharpness indicator, GPC was testified to be effective for restoration of defocus blur image. Since a deconvolution kernel was designed based on Wiener filter, the additive noise was able to be estimated in several flat regions, which are selected by Normalized Total Variation (NTV) measurement. Moreover, according to the unimodal property between GPC of images obtained via deconvolution and estimated defocus blur radius, a "Pyramid" search is adopted to determine the exact blur radius and restore the image. The experimental results reveal that the proposed method was particularly effective for defocus blur image even with additive noises.

Micro et nano-positionnement robuste par l'asservissement visuel Avec le développement des nanotechnologies, il est devenu possible et souhaitable de créer et d'assembler des nano-objets. Afin d'obtenir des processus automatisés robustes et fiables, la manipulation à l'échelle nanométrique est devenue, au cours des dernières années, une tâche primordiale. La vision est un moyen indispensable pour observer le monde à l'échelle micrométrique et nanométrique. Le contrôle basé sur la vision est une solution efficace pour les problèmes de contrôle de la robotique. Dans cette thèse, nous abordons la problématique du micro- et nano-positionnement par asservissement visuel via l'utilisation d'un microscope électronique à balayage (MEB). Dans un premier temps, la formation d'image MEB et les modèles géométriques de la vision appliqués aux MEB sont étudiés afin de présenter, par la suite, une méthode d'étalonnage de MEB par l'optimisation non-linéaire considérant les modèles de projection perspective et parallèle. Dans cette étude, il est constaté qu'il est difficile d'observer l'information de profondeur à partir de la variation de la position de pixel de l'échantillon dans l'image MEB à un grossissement élevé. Afin de résoudre le problème de la non-observabilité du mouvement dans l'axe de la profondeur du MEB, les informations de défocalisation d'image sont considérées comme caractéristiques visuelles pour commander le mouvement sur cet axe. Une méthode d'asservissement visuelle hybride est alors proposée pour effectuer le micro-positionnement en 6 degrés de liberté en utilisant les informations de défocalisation d'image et de photométrique d'image. Cette méthode est ensuite validée via l'utilisation d'un robot parallèle dans un MEB. Finalement, un système de contrôle en boucle fermée pour l'autofocus du MEB est introduit et validé par des expériences. Une méthode de suivi visuel et d'estimation de pose 3D, par la mise en correspondance avec un modèle de texture, est proposée afin de réaliser le guidage visuel dans un MEB. Cette méthode est robuste au flou d'image à cause de la défocalisation provoquée par le mouvement sur l'axe de la profondeur car le niveau de défocalisation est modélisée dans ce cadre de suivi visuel.