As Machine Vision (MV) and Artificial Intelligence (AI) systems are incorporated to an ever-increasing range of imaging applications, there is a corresponding need for camera measurements that can accurately predict the performance of these systems. At the present time, the standard practice is to separately measure the two major factors, sharpness and noise (or Signal-to-Noise Ratio), along with several additional factors, then to estimate system performance based on a combination of these factors. This estimate is usually based on experience, and is often more of an art than a science. Camera information capacity (C), based on Claude Shannon's ground-breaking work on information theory, holds great promise as a figure of merit for a variety of imaging systems, but it has traditionally been difficult to measure. We describe a new method for measuring camera information capacity that uses the popular slanted-edge test pattern, specified by the ISO 12233:2014/2017 standard. Measuring information capacity requires no extra effort: it essentially comes for free with slanted-edge MTF measurements. C has units of bits per pixel or bits per image for a specified ISO speed and chart contrast, making it easy to compare very different cameras. The new measurement can be used to solve some important problems, such as finding a camera that meets information capacity requirements with a minimum number of pixels, important because fewer pixels mean faster processing as well as lower cost.
Optimizing exposure time for low light scenarios involves a trade-off between motion blur and signal to noise ratio. A method for defining the optimum exposure time for a given function has not been described in the literature. This paper presents the design of a simulation of motion blur and exposure time from the perspective of a real-world camera. The model incorporates characteristics of real-world cameras including the light level (quanta), shot noise and lens distortion. In our simulation, an image quality target chart called the Siemens Star chart will be used, and the simulation outputs a blurred image as if captured from a camera of set exposure and set movement speed. The resulting image is then processed in Imatest in which image quality readings will be extracted from the image and consequently the relationship between exposure time, motion blur and the image quality metrics can be evaluated.
Coating a printed surface with a smooth transparent layer can modify its color. This is due to light interreflections within the coating layer which produce a halo-shaped point spread function. The change of color is related to the coating thickness and the halftone screening used for printing. Thanks to an optical model able to predict the spectral reflectance of the coated print from the one of the non-coated print, we propose to study the impact of the halftone pattern (shape and profile) on the color change caused by the coating layer. It was found that line halftone patterns with a crenel profile induces the strongest changes of color. This is therefore the pattern that we use for an innovative application of this phenomenon: revealing a binary image by adding or removing a coating layer on the print that is originally uniform.
The COVID-19 virus induces infection in both the upper respiratory tract and the lungs. Chest X-ray are widely used to diagnose various lung diseases. Considering chest X-ray and CT images, we explore deep-learning-based models namely: AlexNet, VGG16, VGG19, Resnet50, and Resnet101v2 to classify images representing COVID-19 infection and normal health situation. We analyze and present the impact of transfer learning, normalization, resizing, augmentation, and shuffling on the performance of these models. We explored the vision transformer (ViT) model to classify the CXR images. The ViT model incorporates multi-headed attention to disclose more global information in constrast to CNN models at lower layers. This mechanism leads to quantitatively diverse features. The ViT model renders consolidated intermediate representations considering the training data. For experimental analysis, we use two standard datasets and exploit performance metrics: accuracy, precision, recall, and F1-score. The ViT model, driven by self-attention mechanism and longrange context learning, outperforms other models.
Passive stereo vision systems are useful for estimating 3D geometries from digital images similar to the human biological system. In general, two cameras are situated at a known distance from the object and simultaneously capture images of the same scene from different views. This paper presents a comparative evaluation of 3D geometries of scenes estimated by three disparity estimation algorithms, namely the hybrid stereo matching algorithm (HCS), factor graph-based stereo matching algorithm (FGS), and a multi-resolution FGS algorithm (MR-FGS). Comparative studies were conducted using our stereo imaging system as well as hand-held, consumer-market digital cameras and camera phones of a variety of makes/models. Based on our experimental results, the factor graph algorithm (FGS) and multi-resolution factor graph algorithm (MR-FGS) result in a higher level of 3D reconstruction accuracy than the HCS algorithm. When compared with the FGS algorithm, MR-FGS provides a significant improvement in the disparity contrast along the depth boundaries and minimal depth discontinuities.
In recent years, we have seen significant progress in advanced image and video upscaling techniques, sometimes called super-resolution, or AI-based upscaling. Such algorithms are now broadly available in the forms of software SDKs, as well as functions natively supported by modern graphics cards. However, to take advantage of such technologies in video streaming applications, one needs to (a) add support for super-resolution upscaling in the video rendering chain, (b) develop means for quantifying the effects of using different upscaling techniques on perceived quality, and c) modify streaming clients to use such more advanced scaling techniques in a way that leads to improvements in quality, efficiency, or both. In this paper, we discuss several techniques addressing these challenges. We first present an overview of super-resolution technology. We review available SDKs and libraries for adding super-resolution functionality in streaming players. We next propose a parametric quality model suitable for modeling the effects of different upscaling techniques. We validate it by using an existing widely used dataset with subjective scores. And finally, we present an improved adaptation logic for streaming clients, allowing them to save bandwidth while maintaining quality at the level achievable by standard scaling techniques. Our experiments show that this logic can reduce streaming bitrates by up to 38.9%.
Training deep models that can be deployed on embedded systems to robustly detect and track highly specialized industrial objects in a variety of field environments remains very challenging. Large Deep Foundation models (e.g., [yuan21]) make it easier than ever to detect and track everyday objects but do not work as well for specialized industrial objects. These models are often very large and not suitable for deployment on embedded systems. In this work we show that the use of a chroma-key like substitution combined with artificial occlusion generation allows one to capture a small number of images of objects under a fixed background in the lab and then generalize them to novel backgrounds that work in the real world under realistic conditions improving detection of occluded objects by 4% and improving detection in different environments by 44% over state-of-the-art augmentation methods such as MOSAIC.
In the recent years, the detection of deepfakes has become a substantial topic in image and video forensics. State-of-the-art blind detection methods can detect deepfakes from synthetic datasets with high accuracies. However, they struggle to classify deepfake material that underwent adversarial post-processing or fail to generalize to unseen video data. In this paper, a refined detection pipeline taking advantage of a semi-blind detection scheme is proposed. It combines background-matching with a state-of-the-art CNN-classifier. When classifying videos from the Deepfake Detection Challenge Dataset the CNN-classifier was previously trained on, the performance did not improve using the new detection scheme. However, the approach was able to achieve superior results on unseen data of the FaceForensics++ Dataset.
This paper proposes a novel information visualisation interface to help with the reading and improvement of biochips.The interface serves two main groups of end users.These are bio-chip model users and bio-chip model developers.Bio-chip model users are biologists who use the software to read chips and detect biochemical substances.Bio-chip model developers use the software to design and train classification models by seeing how well the different biosensors work and how well the data fits their model.The interface proposed uses a Random Forest classifier and visualises the classification to provide a better understanding of how the data is classified by showing how it fits different classifications and how changes in attribute values can affect the classification.The interface also allows model-developers to interact to see how their model works for different attribute values, and shows them how new data (sent by model-users) fits into their classification model.This allow the biochip designers to detect how their model may be limited so they can retrain the model accordingly.The particular challenge with this project is how we manage and visualise uncertainty related to bio-sensor readings (that can be resultant from the manufacturing process and environmental factors) and the machine learning models, so that biologists can account for this when designing or using chips.Overall, our interface demonstrates the potential of information visualisation to be used to allow developers and model-users to better understand the effectiveness of classification models for their data, as well as the potential of collaborative interfaces to help them work together to build more effective supervised classification models.
Optical flow estimation is a well-studied topic for automated driving applications. Many outstanding optical flow estimation methods have been proposed, but they become erroneous when tested in challenging scenarios that are commonly encountered. Despite the increasing use of fisheye cameras for near-field sensing in automated driving, there is very limited literature on optical flow estimation with strong lens distortion. Thus we propose and evaluate training strategies to improve a learning-based optical flow algorithm by leveraging the only existing fisheye dataset with optical flow ground truth. While trained with synthetic data, the model demonstrates strong capabilities to generalize to real world fisheye data. The other challenge neglected by existing state-of-the-art algorithms is low light. We propose a novel, generic semi-supervised framework that significantly boosts performances of existing methods in such conditions. To the best of our knowledge, this is the first approach that explicitly handles optical flow estimation in low light.
The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in daily life. Mobile devices and applications provide many benefits for users, perhaps most significantly is the increased access to point-of-use tools, navigation, and alert systems. This paper presents a prototype of a cross-platform mobile augmented reality (AR) system with the core purpose of finding a better means to keep the campus community secure and connected. The mobile AR System consists of four core functionalities – an events system, a policing system, a directory system, and a notification system. The events system keeps the community up-to-date on current events that are happening or will be happening on campus. The policing system allows the community to stay in arms reach of campus resources that will allow them to stay secure. The directory system serves as a one-stop-shop for campus resources, ensuring that staff, faculty, and students will have a convenient and efficient means of accessing pertinent information on the campus departments. The mobile augmented reality system includes integrated guided navigation system that users can use to get directions to various destinations on campus. The various destinations are different buildings and departments on campus. This mobile augmented reality application will assist the students and visitors on campus to efficiently navigate the campus as well as send alert and notifications in case of emergencies. This will allow campus police to respond to the emergencies in a quick and timely manner. The mobile AR system was designed using Unity Game Engine and Vuforia Engine for object detection and classification. Google Map API was integrated for GPS integration in order to provide location-based services. Our contribution lies in our approach to create a user specific customizable navigational and alert system in order to improve the safety of the users at their workplace. Specifically, the paper describes the design and implementation of the proposed mobile AR system and reports the results of the pilot study conducted to evaluate their perceived ease-of-use, and usability.
Coherent anti-Stokes Raman scattering (CARS) microspectroscopy is a powerful tool for label-free cell imaging thanks to its ability to acquire a rich amount of information. An important family of operations applied to such data is multivariate curve resolution (MCR). It aims to find main components of a dataset and compute their spectra and concentrations in each pixel. Recently, autoencoders began to be studied to accomplish MCR with dense and convolutional models. However, many questions, like the results variability or the reconstruction metric, remain open and applications are limited to hyperspectral imaging. In this article, we present a nonlinear convolutional encoder combined with a linear decoder to apply MCR to CARS microspectroscopy. We conclude with a study of the result variability induced by the encoder initialization.
A reset noise reduction method using a feedback amplifier that results in an 80% noise reduction in 3-transistor (3-T) pixels is presented. 3-T pixels are useful for non-visible imaging applications because they have fewer post-processing issues than 4-T pixels and do not require charge transfer. They suffer from reset noise because correlated-double sampling cannot be realized without additional memory. Analysis of the experimental power spectral density indicates potential for further noise cancellation in future devices.
Due to the emergence of machine vision, augmented reality (AR), virtual reality (VR), and automotive connectivity in recent years, the necessity for chip miniaturization has grown. These emerging, next-generation applications, which are centered on user experience and comfort, require their constituent chips, devices, and parts to be smaller, lighter, and more accessible. AR/VR applications, especially demand smaller components due to their primary application towards wearable technology, in which the user experience would be negatively impacted by large features and bulk. Therefore, chips and devices intended for next-generation consumer applications must be small and modular, to support module miniaturization and promote user comfort. To enable the chip miniaturization required for technological advancement and innovation, we developed a 2.2μm pixel pitch Back Side Illuminated (BSI) Voltage Domain Global Shutter (VDGS) image sensor with the three-wafer stacked technology. Each wafer is connected by Stacked Pixel Level Connection (SPLC) and the middle and logic wafers are connected using a Back side Through Silicon Via (BTSV). The separation of the sensing, charge storage, and logic functions to different wafers allows process optimization in each wafer, improving overall chip performance. The peripheral circuit region is reduced by 75% compared to the previous product without degrading image sensor performance. For the session: Processing at the Edge (joint with COIMG).
The iridescence effect, produced by structural color, is difficult (if not impossible) to capture and print using traditional CMYK pigments. RGB pigments, nonetheless, generate structural colors by light interference. The layered surface structure generated by pigments’ particles reflects different wavelengths of light in different viewing angles. In printed media, pigments’ particles will collectively influence the optical response of the surface, depending on their size, orientation, structure, and dimensions, ultimately, affecting the visual characteristics of the image perceived by the observer. In this work, we have studied the influence of different halftones’ structures on printed images, produced with RGB inks via screen printing. We investigated the influence of different halftones’ structures in creating different spatial combinations of inks on the printed surface that reproduce the characteristics of iridescent effect of a headdress made of quetzal feathers. We applied first-order, second-order, and structure-aware FM halftones to compare how they influence the reproduction of the material qualities of the object represented in the image. The results show that the structure-ware halftones improve the representation of the image structures and details and, therefore, it could better convey the 3D surface features that produce iridescence effect in the original feathers of the headdress.
The COVID-19 epidemic has been a significant healthcare challenge in the United States. According to the Centers for Disease Control and Prevention (CDC), COVID-19 infection is transmitted predominately by respiratory droplets generated when people breathe, talk, cough, or sneeze. Wearing a mask is the primary, effective, and convenient method of blocking 80% of all respiratory infections. Therefore, many face mask detection and monitoring systems have been developed to provide effective supervision for hospitals, airports, publication transportation, sports venues, and retail locations. However, the current commercial face mask detection systems are typically bundled with specific software or hardware, impeding public accessibility. In this paper, we propose an in-browser serverless edge-computing based face mask detection solution, called Web-based efficient AI recognition of masks (WearMask), which can be deployed on any common devices (e.g., cell phones, tablets, computers) that have internet connections using web browsers, without installing any software. The serverless edge-computing design minimizes the extra hardware costs (e.g., specific devices or cloud computing servers). The contribution of the proposed method is to provide a holistic edge-computing framework of integrating (1) deep learning models (YOLO), (2) high-performance neural network inference computing framework (NCNN), and (3) a stack-based virtual machine (WebAssembly). For end-users, our web-based solution has advantages of (1) serverless edge-computing design with minimal device limitation and privacy risk, (2) installation free deployment, (3) low computing requirements, and (4) high detection speed. Our WearMask application has been launched with public access at facemask-detection.com.
Infrastructure maintenance of complex environments like railroads is a very expensive operation. Recent advances in mobile mapping systems to collect 3D point cloud data and in deep learning for detection and segmentation can prove to be very helpful in automating this maintenance and allowing preventive maintenance at certain locations before big failures occur. Some fully-supervised methods have been developed for understanding dynamic railroad environments. These methods often fail to generalize to infrastructure changes or new classes in low-labeled data. To address this issue, we propose a railroad segmentation method that leverages few-shot learning by generating class prototypes for the most relevant infrastructure classes. This method takes advantage of existing embedding networks for point clouds, taking the geometrical and spatial context into account for feature representation of complex connected classes. We evaluate our method on real-world data measured on Belgian railway tracks. Our model achieves promising results on connected classes, exposed to only a few annotated samples at test time.
Care experiences and health outcomes may suffer greatly because of healthcare professionals' deficient educational preparation and practices. The limited awareness about the impact of stereotypes, implicit/explicit biases, and social determinants of health (SDH) may result in unpleasant care experiences and healthcare professional-patient relationships. Additionally, as healthcare professionals are no less prone to have biases than other people, it is essential to deliver the learning platform to enhance healthcare skills (e.g., awareness of the importance of cultural humility, inclusive communication proficiencies, awareness of the enduring impact of both SDH and implicit/explicit biases on health outcomes, and compassionate and empathetic attitude) of healthcare professionals, which eventually help to raise health equity in society. Moreover, employing the "learning-by-doing" approach directly in real-life clinical practices is less preferable wherein high-risk care is essential. Thus, there is a huge scope to deliver virtual reality-based care practices by engaging the digital experiential learning and human-computer interaction (HCI) approach to enhance patient care experiences, healthcare experiences, and healthcare skills. Thus, this research provides the computer-supported experiential learning (CSEL) approach-based tool or mobile application that facilitates virtual reality-based serious role-playing scenarios to enhance the healthcare skills of healthcare professionals and for public awareness.
Driving assistance is increasingly used in new car models. Most driving assistance systems are based on automotive cameras and computer vision. Computer Vision, regardless of the underlying algorithms and technology, requires the images to have good image quality, defined according to the task. This notion of good image quality is still to be defined in the case of computer vision as it has very different criteria than human vision: humans have a better contrast detection ability than image chains. The aim of this article is to compare three different metrics designed for detection of objects with computer vision: the Contrast Detection Probability (CDP) [1, 2, 3, 4], the Contrast Signal to Noise Ratio (CSNR) [5] and the Frequency of Correct Resolution (FCR) [6]. For this purpose, the computer vision task of reading the characters on a license plate will be used as a benchmark. The objective is to check the correlation between the objective metric and the ability of a neural network to perform this task. Thus, a protocol to test these metrics and compare them to the output of the neural network has been designed and the pros and cons of each of these three metrics have been noted.
The edge-based Spatial Frequency Response (e-SFR) method was first developed for evaluating camera image resolution and image sharpness. The method was described in the first version of the ISO 12233 standard. Since then, the method has been applied in a wide range of applications, including medical, security, archiving, and document processing. However, with this broad application, several of the assumptions of the method are no longer closely followed. This has led to several improvements aimed at broadening its application, for example for lenses with spatial distortion. We can think of the evaluation of image quality parameters as an estimation problem, based on the gathered data, often from digital images. In this paper, we address the mitigation of measurement error that is introduced when the analysis is applied to low-exposure (and therefore, noisy) applications and those with small analysis regions. We consider the origins of both bias and variation in the resulting SFR measurement and present practical ways to reduce them. We describe the screening of outlier edge-location values as a method for improved edge detection. This, in turn, is related to a reduction in negative bias in the resulting SFR.
