Nanoscale secondary ion mass spectrometry (nanoSIMS) and fluorescence in situ hybridization (FISH) microscopy provide high-resolution, multimodal image representations of the identity and cell activity respectively of targeted microbial communities in microbiological research. Despite its importance to microbiologists, multimodal registration of FISH and nanoSIMS images is challenging given the morphological distortion and background noise in both images. In this study, we use convolutional neural networks (CNNs) for multiscale feature extraction, shape context for computation of the minimum transformation cost feature matching and the thin-plate spline (TPS) model for multimodal registration of the FISH and nanoSIMS images. Registration accuracy was quantitatively assessed against manually registered images, at both, the pixel and structural levels using standard metrics. Although all six tested CNN models performed well, ResNet18 was observed to outperform VGG16 , VGG19 , GoogLeNet and ShuffleNet and ResNet101 based on most metrics. This study demonstrates the utility of CNNs in the registration of multimodal images with significant background noise and morphology distortion. We also show aggregate shape, preserved by binarization, to be a robust feature for registering multimodal microbiology-related images.
The impressive rise of Deep Learning and, more specifically, the discovery of generative adversarial networks has revolutionised the world of Deepfake. The forgeries are becoming more and more realistic, and consequently harder to detect. Attesting whether a video content is authentic is increasingly sensitive. Furthermore, free access to forgery technologies is dramatically increasing and very worrying. Numerous methods have been proposed to detect these deepfakes and it is difficult to know which detection methods are still accurate regarding the recent advances. Therefore, an approach for face swapping detection in videos, based on residual signal analysis is presented in this paper.
Rapidly emerging augmented reality technologies enable us to virtually alter appearance of objects and materials in a fast and efficient way. The state-of-the-art research shows that the human visual system has a poor ability to invert the optical processes in the scene and rather relies on images cues and spatial distribution of luminance to perceive appearance attributes, such as gloss and translucency. For this reason, we hypothesize that it is possible to alter gloss and translucency appearance by projecting an image onto the original to mimic the luminance distribution characteristic of the target appearance. To demonstrate feasibility of this approach, we use pairs of physically-based renderings of glossy and matte, and translucent and opaque materials, respectively; we calculate a compensation image – a luminance difference between them, and subsequently, we demonstrate that by algebraic addition of luminance, an image of matte object can appear glossy, and an image of opaque object can appear translucent, when respective compensation images are projected onto them. Furthermore, we introduce a novel method to increase apparent opacity of translucent materials. Finally, we propose a future direction, which could enable nearly real-time appearance manipulation.
In this paper we investigate the possibility of constructing 3D models of longer sections of the human colon using image sequences obtained from wireless capsule endoscope (WCE) video to provide enhanced viewing for gastroenterologists. As images from WCE contain severe distortions and artifacts non-ideal for 3D reconstruction algorithms, the problem is difficult to attack. However, recent developments of virtual graphics-based models of human gastrointestinal system, where most of the distortions and artifacts can be enabled or disabled, makes it possible to determine how each factor disturbs such algorithms individually. In this paper we disable distortions and artifacts in order to determine if longer sections of the human intestinal environment is at all feasible to reconstruct. Though simulation we show that this is possible using structure from motion and simultaneous localization and mapping (SLAM).
While transmittance is a physically measurable quantity, people perceive it as the translucency of object surfaces. However, transmittance does not always match translucency. We measured the physical properties of object surfaces, including physical transmittance, and analyzed the relationship between the physical properties and translucency. We prepared 107 samples of flat objects that primarily consisted of resin for the experiment. We visually evaluated the perpetual gloss using a magnitude estimation method. We conducted multiple measurements of physical properties such as transmittance, haze, distinctness of image, and gloss unit. Then we constructed a prediction model for evaluating the perceptual gloss using the abovementioned physical properties and translucency through multiple regression analysis. As a result, the prediction accuracy is found to be improved by combining various physical quantities with simple regression using transmittance.
Although a typical display consists of red-green-blue (RGB) subpixels, displays with various subpixel layouts have been used owing to their various advantages such as luminous efficiency. The purpose of this study is to verify the effects of display MTF on subjective spatial resolution by creating new subpixel layouts. We designed BRGRB and BRGRB525 subpixel layouts with slightly higher and much higher MTF than RGB subpixel layouts, and conducted visual evaluation experiments with RGB, PenTile RGBG and the two new subpixel layouts at 20 and 30 cycles per degree. It was verified that subjective spatial resolution generally follows the large and small relationship of display MTF. Additional experiments showed that the integral of the product of the contrast sensitivity function and MTF was highly correlated with the subjective spatial resolution.
Subjective ratings given by observers are a critical part of research in image and video quality assessment. Like any other field of science, with subjective data collection, researchers may lack the expertise needed to address the different issues they face. In this study, we review different approaches and find potential pitfalls that generally seem overlooked in quality research. To address these issues, we found six relevant pitfalls relating to recruitment, instructions, experimental design, and data analysis that could be addressed by studies done in the field of cognitive science. Combining accessed datasets from quality research with newly collected data, we statistically demonstrated four of the six pitfalls: observers used the scale non-linearly; ratings can change throughout the experiment; features can influence individual observers differently; and allowing observers to decide how many ratings they give can lead to biases. We need additional data to investigate the two pitfalls related to instructions and recruitment. Our findings suggest that pitfalls which might not be initially clear to researchers in the field of image and video processing can still have an empirically demonstrable influence on the data. While this article will not solve every issue, it will try to suggest improvements that researchers can readily employ.
It is well-known that the varnish applied to artwork yellows with time changing its appearance accordingly. Conservators are then sometimes prompted to physically clean the artwork in an attempt to recover the original look of the work. At times, the conservators only partially clean the artwork first. They then try to virtually clean the rest of the artwork to visualize the result of the cleaning before physically cleaning the entire piece. There have been many different approaches that have been proposed to virtually clean a partially cleaned artwork. All of them have some limitations, the low accuracy of which is the main one. In this paper, a deep generative network is proposed to virtually clean a partially cleaned artwork in the RGB domain. The proposed generative model consists of several up-sampling and down-sampling convolution blocks and skip connections with a symmetric architecture. The loss function is calculated using the part of the artwork that has been physically cleaned for which we have access to both RGB images before and after cleaning. Therefore the network is able to clean the whole artwork using only a small area of it that has already been physically cleaned. A Macbeth ColorChecker and images of the Mona Lisa are used to test the approach and the results are compared with a recent approach available in the literature which uses a Convolutional Neural Network (CNN). The results are found to be acceptable given that the approach proposed herein has a potential to be applied in a real situation and there is no need for a large training dataset, on which the CNN method relied on.
Intrinsically photosensitive retinal ganglion cells (ipRGCs) affect pupillary light reflex and circadian rhythm regulation. Recent studies have reported that they affect visual perception, particularly brightness perception, and their effect on color perception has also been gradually reported. In this study, we performed a color-matching task on a display device to verify the effect of ipRGC on color perception. Over a year, one participant performed 310 color matching sessions, day and night, by central and peripheral vision. Three blue colors with high ipRGC absorption and three red colors with low ipRGC absorption were used in one session. The color matching results suggest that non-image-forming and image-forming functions interact with col-or perception. We built a regression model in which ipRGC acts on the LMS. We found that the models constructed for each hue explained the experimental results well.
Dye-sublimation printing is a modern digital printing technology that is being used more and more for textile printing. Classifying and identifying ink produced by the different manufacturers can be important to understand and analyse issues like document forgery, poor print quality, and printer service needs in the market. In this paper, we classify and evaluate the performance of dye-sublimation printing inks produced by six different manufacturers. The inks are evaluated in terms of colour and reproduction accuracy on two different textile materials. A weighting coefficient is introduced in the classification threshold to consider different customer requirements like quality, application, and process standardisation during classification. The obtained results indicate that with sufficient spectral measurement data, spectral angle calculated using the spectral angle mapper algorithm and the Mahalanobis distance between the inks, this would be possible.
