There can be numerous electronic components on a given PCB, making the task of visual inspection to detect defects very time-consuming and prone to error, especially at scale. There has thus been significant interest in automatic PCB component detection, particularly leveraging deep learning. However, deep neural networks typically require high computational resources, possibly limiting their feasibility in real-world use cases in manufacturing, which often involve high-volume and high-throughput detection with constrained edge computing resource availability. As a result of an exploration of efficient deep neural network architectures for this use case, we introduce PCBDet, an attention condenser network design that provides state-of-the-art inference throughput while achieving superior PCB component detection performance compared to other state-of-the-art efficient architecture designs. Experimental results show that PCBDet can achieve up to 2$\times$ inference speed-up on an ARM Cortex A72 processor when compared to an EfficientNet-based design while achieving $\sim$2-4\% higher mAP on the FICS-PCB benchmark dataset.

Vision transformers have shown unprecedented levels of performance in tackling various visual perception tasks in recent years. However, the architectural and computational complexity of such network architectures have made them challenging to deploy in real-world applications with high-throughput, low-memory requirements. As such, there has been significant research recently on the design of efficient vision transformer architectures. In this study, we explore the generation of fast vision transformer architecture designs via generative architecture search (GAS) to achieve a strong balance between accuracy and architectural and computational efficiency. Through this generative architecture search process, we create TurboViT, a highly efficient hierarchical vision transformer architecture design that is generated around mask unit attention and Q-pooling design patterns. The resulting TurboViT architecture design achieves significantly lower architectural computational complexity (>2.47$\times$ smaller than FasterViT-0 while achieving same accuracy) and computational complexity (>3.4$\times$ fewer FLOPs and 0.9% higher accuracy than MobileViT2-2.0) when compared to 10 other state-of-the-art efficient vision transformer network architecture designs within a similar range of accuracy on the ImageNet-1K dataset. Furthermore, TurboViT demonstrated strong inference latency and throughput in both low-latency and batch processing scenarios (>3.21$\times$ lower latency and >3.18$\times$ higher throughput compared to FasterViT-0 for low-latency scenario). These promising results demonstrate the efficacy of leveraging generative architecture search for generating efficient transformer architecture designs for high-throughput scenarios.

Multi-task learning has shown considerable promise for improving the performance of deep learning-driven vision systems for the purpose of robotic grasping. However, high architectural and computational complexity can result in poor suitability for deployment on embedded devices that are typically leveraged in robotic arms for real-world manufacturing and warehouse environments. As such, the design of highly efficient multi-task deep neural network architectures tailored for computer vision tasks for robotic grasping on the edge is highly desired for widespread adoption in manufacturing environments. Motivated by this, we propose Fast GraspNeXt, a fast self-attention neural network architecture tailored for embedded multi-task learning in computer vision tasks for robotic grasping. To build Fast GraspNeXt, we leverage a generative network architecture search strategy with a set of architectural constraints customized to achieve a strong balance between multitask learning performance and embedded inference efficiency. Experimental results on the MetaGraspNet benchmark dataset show that the Fast GraspNeXt network design achieves the highest performance (average precision (AP), accuracy, and mean squared error (MSE)) across multiple computer vision tasks when compared to other efficient multi-task network architecture designs, while having only 17.8M parameters (about >5× smaller), 259 GFLOPs (as much as >5× lower) and as much as >3.15× faster on a NVIDIA Jetson TX2 embedded processor.

Modern deep neural networks must demonstrate state-of-the-art accuracy while exhibiting low latency and energy consumption. As such, neural architecture search (NAS) algorithms take these two constraints into account when generating a new architecture. However, efficiency metrics such as latency are typically hardware dependent requiring the NAS algorithm to either measure or predict the architecture latency. Measuring the latency of every evaluated architecture adds a significant amount of time to the NAS process. Here we propose Microprocessor A Priori for Latency Estimation (MAPLE) that leverages hardware characteristics to predict deep neural network latency on previously unseen hardware devices. MAPLE takes advantage of a novel quantitative strategy to characterize the underlying microprocessor by measuring relevant hardware performance metrics, yielding a fine-grained and expressive hardware descriptor. The CPU-specific performance metrics are also able to characterize GPUs, resulting in a versatile descriptor that does not rely on the availability of hardware counters on GPUs or other deep learning accelerators. We provide experimental insight into this novel strategy. Through this hardware descriptor, MAPLE can generalize to new hardware via a few shot adaptation strategy, requiring as few as 3 samples from the target hardware to yield 6% improvement over state-of-the-art methods requiring as much as 10 samples. Experimental results showed that, increasing the few shot adaptation samples to 10 improves the accuracy significantly over the state-of-the-art methods by 12%. We also demonstrate MAPLE identification of Pareto-optimal DNN architectures exhibit superlative accuracy and efficiency. The proposed technique provides a versatile and practical latency prediction methodology for DNN run-time inference on multiple hardware devices while not imposing any significant overhead for sample collection.

Neural Architecture Search (NAS) has enabled automatic discovery of more efficient neural network architectures, especially for mobile and embedded vision applications. Although recent research has proposed ways of quickly estimating latency on unseen hardware devices with just a few samples, little focus has been given to the challenges of estimating latency on runtimes using optimized graphs, such as TensorRT and specifically for edge devices. As devices like NVIDIA’s Jetsons get more popular in embedded computing and robotics, we observe a pressing need to more accurately estimate inference latency of neural network architectures on diverse runtimes, including highly optimized ones. In this work, we propose MAPLE-Edge, an edge device-oriented extension of MAPLE, the state-of-the-art latency predictor for general purpose hardware, where we train a regression network on architecture-latency pairs in conjunction with a hardware-runtime descriptor to effectively estimate latency on a diverse pool of edge devices. Compared to MAPLE, MAPLE-Edge can describe the runtime and target device platform using a much smaller set of CPU performance counters that are widely available on all Linux kernels, while still achieving up to +49.6% accuracy gains against previous state-of-the-art baseline methods on optimized edge device runtimes, using just 10 measurements from an unseen target device. We also demonstrate that unlike MAPLE which performs best when trained on a pool of devices sharing a common runtime, MAPLE-Edge can effectively generalize across runtimes by applying a trick of normalizing performance counters by the operator latency, in the measured hardware-runtime descriptor. Lastly, we show that for runtimes exhibiting lower than desired accuracy, performance can be boosted by collecting additional samples from the target device, with an extra 90 samples translating to gains of nearly +40%.

Deep neural network (DNN) latency characterization is a time-consuming process and adds significant cost to Neural Architecture Search (NAS) processes when searching for efficient convolutional neural networks for embedded vision applications. DNN Latency is a hardware dependent metric and requires direct measurement or inference on target hardware. A recently introduced latency estimation technique known as MAPLE predicts DNN execution time on previously unseen hardware devices by using hardware performance counters. Leveraging these hardware counters in the form of an implicit prior, MAPLE achieves state-of-the-art performance in latency prediction. Here, we propose MAPLE-X which extends MAPLE by incorporating explicit prior knowledge of hardware devices and DNN architecture latency to better account for model stability and robustness. First, by identifying DNN architectures that exhibit a similar latency to each other, we can generate multiple virtual examples to significantly improve the accuracy over MAPLE. Secondly, the hardware specifications are used to determine the similarity between training and test hardware to emphasize training samples captured from comparable devices (domains) and encourages improved domain alignment. Experimental results using a convolution neural network NAS benchmark across different types of devices, including an Intel processor that is now used for embedded vision applications, demonstrate a 5% improvement over MAPLE and 9% over HELP. Furthermore, we include ablation studies to independently assess the benefits of virtual examples and hardware-based sample importance.

With the growing adoption of deep learning for on-device TinyML applications, there has been an ever-increasing demand for efficient neural network backbones optimized for the edge. Recently, the introduction of attention condenser networks have resulted in low-footprint, highly-efficient, self-attention neural networks that strike a strong balance between accuracy and speed. In this study, we introduce a faster attention condenser design called double-condensing attention condensers that allow for highly condensed feature embeddings. We further employ a machine-driven design exploration strategy that imposes design constraints based on best practices for greater efficiency and robustness to produce the macro-micro architecture constructs of the backbone. The resulting backbone (which we name AttendNeXt) achieves significantly higher inference throughput on an embedded ARM processor when compared to several other state-of-the-art efficient backbones (>10x faster than FB-Net C at higher accuracy and speed and >10x faster than MobileOne-S1 at smaller size) while having a small model size (>1.37x smaller than MobileNetv3-L at higher accuracy and speed) and strong accuracy (1.1% higher top-1 accuracy than MobileViT XS on ImageNet at higher speed). These promising results demonstrate that exploring different efficient architecture designs and self-attention mechanisms can lead to interesting new building blocks for TinyML applications.

As the COVID-19 pandemic continues to put a significant burden on healthcare systems worldwide, there has been growing interest in finding inexpensive symptom pre-screening and recommendation methods to assist in efficiently using available medical resources such as PCR tests. In this study, we introduce the design of COVID-Net Assistant, an efficient virtual assistant designed to provide symptom prediction and recommendations for COVID-19 by analyzing users' cough recordings through deep convolutional neural networks. We explore a variety of highly customized, lightweight convolutional neural network architectures generated via machine-driven design exploration (which we refer to as COVID-Net Assistant neural networks) on the Covid19-Cough benchmark dataset. The Covid19-Cough dataset comprises 682 cough recordings from a COVID-19 positive cohort and 642 from a COVID-19 negative cohort. Among the 682 cough recordings labeled positive, 382 recordings were verified by PCR test. Our experimental results show promising, with the COVID-Net Assistant neural networks demonstrating robust predictive performance, achieving AUC scores of over 0.93, with the best score over 0.95 while being fast and efficient in inference. The COVID-Net Assistant models are made available in an open source manner through the COVID-Net open initiative and, while not a production-ready solution, we hope their availability acts as a good resource for clinical scientists, machine learning researchers, as well as citizen scientists to develop innovative solutions.

Here we explore the use of Photoacoustic Remote Sensing (PARS™) microscopy, a recently developed non-contact photoacoustic imaging modality, for visualizing subcellular structures label-free in tissues. Operating in an all-optical reflection-mode architecture PARS captures optical absorption contrast within bulk tissue samples. Presented here, by visualizing endogenous optical absorption of DNA and cytochromes, cellular morphology is captured with contrast analogous to the industry standard hematoxylin and eosin (H&E) staining. Subcellular features are recovered from human and murine, brain and gastrointestinal tissues. This work represents a significant step towards the development of a real-time microscopy system for label-free histopathological assessment of tissues in-situ.

As the COVID-19 pandemic continues to devastate globally, one promising field of research is machine learning-driven computer vision to streamline various parts of the COVID-19 clinical workflow. These machine learning methods are typically stand-alone models designed without consideration for the integration necessary for real-world application workflows. In this study, we take a machine learning and systems (MLSys) perspective to design a system for COVID-19 patient screening with the clinical workflow in mind. The COVID-Net system is comprised of the continuously evolving COVIDx dataset, COVID-Net deep neural network for COVID-19 patient detection, and COVID-Net S deep neural networks for disease severity scoring for COVID-19 positive patient cases. The deep neural networks within the COVID-Net system possess state-of-the-art performance, and are designed to be integrated within a user interface (UI) for clinical decision support with automatic report generation to assist clinicians in their treatment decisions.

Gastrointestinal (GI) tissue biopsies provide critical diagnostic information for a wide variety of conditions such as neoplastic diseases (colorectal, small bowel and stomach cancers) and non-neoplastic diseases (inflammatory disorders, infection, celiac disease). Endoscopic biopsies collect small tissue samples that require resource intensive processing to permit histopathological analysis. Unfortunately, the sparsely collected biopsy samples may fail to capture the pathologic condition because selection of biopsy sites relies on macroscopic superficial tissue features and clinician judgement. Here, we present the first all-optical non-contact label-free non-interferometric photoacoustic microscopy system capable of performing "virtual biopsies". A modular photoacoustic remote sensing (PARS™) architecture is used facilitating imaging of unprocessed tissues providing information similar to conventional histopathological staining techniques. Prospectively this would allow gastroenterologists to assess subcellular tissue morphology in situ when selecting biopsy location. Tested on preserved unstained human and freshly resected murine tissues, the presented PARS microscope rapidly retrieves images of similar area to current biopsies, while maintaining comparable quality to the current standard for histopathological analysis. Additionally, results show the first label free assessment of subsurface cellular morphology in FFPE GI tissue blocks. Clinically relevant features are recovered including cellular details such as lamina propria within colon tissue and cell nuclear structure in resected smooth muscle. Constructed with a modular architecture, this system facilitates the future development of compact imaging heads. The modular PARS system overcomes many of the challenges with imaging unstained thick tissue in situ, representing a significant milestone in the development of a clinical microscope providing virtual biopsy capabilities.

The fine-grained relationship between form and function with respect to deep neural network architecture design and hardware-specific acceleration is one area that is not well studied in the research literature, with form often dictated by accuracy as opposed to hardware function. In this study, a comprehensive empirical exploration is conducted to investigate the impact of deep neural network architecture design on the degree of inference speedup that can be achieved via hardware-specific acceleration. More specifically, we empirically study the impact of a variety of commonly used macro-architecture design patterns across different architectural depths through the lens of OpenVINO microprocessor-specific and GPU-specific acceleration. Experimental results showed that while leveraging hardware-specific acceleration achieved an average inference speed-up of 380%, the degree of inference speed-up varied drastically depending on the macro-architecture design pattern, with the greatest speedup achieved on the depthwise bottleneck convolution design pattern at 550%. Furthermore, we conduct an in-depth exploration of the correlation between FLOPs requirement, level 3 cache efficacy, and network latency with increasing architectural depth and width. Finally, we analyze the inference time reductions using hardware-specific acceleration when compared to native deep learning frameworks across a wide variety of hand-crafted deep convolutional neural network architecture designs as well as ones found via neural architecture search strategies. We found that the DARTS-derived architecture to benefit from the greatest improvement from hardware-specific software acceleration (1200%) while the depthwise bottleneck convolution-based MobileNet-V2 to have the lowest overall inference time of around 2.4 ms.

Human operators often diagnose industrial machinery via anomalous sounds. Given the new advances in the field of machine learning, automated acoustic anomaly detection can lead to reliable maintenance of machinery. However, deep learning-driven anomaly detection methods often require an extensive amount of computational resources prohibiting their deployment in factories. Here we explore a machine-driven design exploration strategy to create OutlierNets, a family of highly compact deep convolutional autoencoder network architectures featuring as few as 686 parameters, model sizes as small as 2.7 KB, and as low as 2.8 million FLOPs, with a detection accuracy matching or exceeding published architectures with as many as 4 million parameters. The architectures are deployed on an Intel Core i5 as well as a ARM Cortex A72 to assess performance on hardware that is likely to be used in industry. Experimental results on the model’s latency show that the OutlierNet architectures can achieve as much as 30× lower latency than published networks.

Nonmelanoma skin cancers (NSMC) are among the most common malignancies in the US today. Mohs micrograph surgery (MMS) is the gold standard for most NSMC. However, MMS is time-consuming as it employs frozen section analysis (FSA) for intraoperative assessment. Each FSA can require up to 60 minutes per excision. Using photoacoustic remote sensing (PARS™) we demonstrate first results of imaging tissue morphology on human skin with a non-contact reflection-mode method, enabling rapid label-free pathological assessment. These images are validated against toluidine blue stained sections. The authors believe the proposed method represents a vital step towards an in-situ assessment of NSMC.

Background: As photoacoustic (PA) techniques progress towards clinical adoption, providing a high-speed live feedback becomes a high priority. To keep up with the instantaneous optical feedback of conventional light microscopes, PA imaging would need to provide a high-resolution video-rate live feed to the user. However, conventional PA microscopy typically trades resolution, sensitivity and imaging speed when optically scanning due to the difficult opto-acoustic confocal geometry. Here, we employ photoacoustic remote sensing (PARS), an all-optical technique that relies on optical confocal geometry, to provide a high resolution live display in a reflection-mode PA architecture. Methods: Employing a conventional x-y galvanometer scanner and a 600 KHz pulse repetition rate laser we implement a system capable of acquiring 2.5 frames per second in 2D. To complement this fast scanning optical system, we implement a computationally inexpensive image reconstruction method that is able to render the frames with minimal overhead, providing a live display. Results: Employing the proposed method, we demonstrate a live feedback with frame rates as high as 2.5 Hz in 2D and also report the first results of 3D imaging with a non-contact label-free reflection-mode technique. The method is validated with phantom studies and in-vivo imaging. Employing a repetition rate of 600 KHz, a live feed of carbon fibers is realized with a C-scan rate of 2.5 Hz. The imaging resolution was measured to be 1.2 mu m, the highest reported for a real-time reflection-mode architecture. The mean and peak SNR were measured to be 44 and 62 dB respectively in-vivo. 3D visualizations of carbon fiber phantoms and mouse ear microvasculature structure are also demonstrated. Conclusions: In summary, we present a method that has a small computational overhead for image rendering, resulting in a live display capable of real-time frame rates. We also report the first 3D imaging with a non-contact label-free reflection-mode PA technique. The all-optical confocal geometry required by PARS is significantly easier to implement and maintain than the opto-acoustic geometry of conventional PA microscopy techniques. This results in a system capable of high resolution and sensitivity, imaging at realtime rates. The authors believe this work represents a vital step towards a clinical high-resolution reflection mode video-rate PA imaging system.

The health and socioeconomic difficulties caused by the COVID-19 pandemic continues to cause enormous tensions around the world. In particular, this extraordinary surge in the number of cases has put considerable strain on health care systems around the world. A critical step in the treatment and management of COVID-19 positive patients is severity assessment, which is challenging even for expert radiologists given the subtleties at different stages of lung disease severity. Motivated by this challenge, we introduce COVID-Net CT-S, a suite of deep convolutional neural networks for predicting lung disease severity due to COVID-19 infection. More specifically, a 3D residual architecture design is leveraged to learn volumetric visual indicators characterizing the degree of COVID-19 lung disease severity. Experimental results using the patient cohort collected by the China National Center for Bioinformation (CNCB) showed that the proposed COVID-Net CT-S networks, by leveraging volumetric features, can achieve significantly improved severity assessment performance when compared to traditional severity assessment networks that learn and leverage 2D visual features to characterize COVID-19 severity.

Histological visualizations are critical to clinical disease management and are fundamental to biological understanding. However, current approaches that rely on bright-field microscopy require extensive tissue preparation prior to imaging. These processes are both labor intensive and contribute to creating significant delays in clinical feedback for treatment decisions that can extend to 2–3 weeks for standard paraffin-embedded tissue preparation and interpretation, especially if ancillary testing is needed. Here, we present the first comprehensive study on the broad application of a novel label-free reflection-mode imaging modality known as photoacoustic remote sensing (PARS) for visualizing salient subcellular structures from various common histopathological tissue preparations and for use in unprocessed freshly resected tissues. The PARS modality permits non-contact visualizations of intrinsic endogenous optical absorption contrast to be extracted from thick and opaque biological targets with optical resolution. The technique was examined both as a rapid assessment tool that is capable of managing large samples (> 1 cm 2 ) in under 10 min, and as a high contrast imaging modality capable of extracting specific biological contrast to simulate conventional histological stains such as hematoxylin and eosin (H&E). The capabilities of the proposed method are demonstrated in a variety of human tissue preparations including formalin-fixed paraffin-embedded tissue blocks and unstained slides sectioned from these blocks, including normal and neoplastic human brain, and breast epithelium involved with breast cancer. Similarly, PARS images of human skin prepared by frozen section clearly demonstrated basal cell carcinoma and normal human skin tissue. Finally, we imaged unprocessed murine kidney and achieved histologically relevant subcellular morphology in fresh tissue. This represents a vital step towards an effective real-time clinical microscope that overcomes the limitations of standard histopathologic tissue preparations and enables real-time pathology assessment.

A multi-wavelength imaging technique based on the principle of stimulated Raman scattering (SRS) can be applied to photoacoustic (PA) techniques to produce label-free image contrast in wavelength-dependent targets in biological tissue. Current studies had limited approaches to optimizing the pulse energy of generated peaks. A comprehensive study of various parameters that affect the pulse energy of generated SRS peaks from a 532nm pulsed laser is presented; including fiber polarization, cut-off wavelength, length of fiber, pulse width, pulse repetition rate, and input power. Optimal conditions for designing a multi-wavelength laser source to image wavelength-dependent biological tissues with possible biomedical diagnostics and experimental applications are presented.

A remote-sensing hyperspectral photoacoustic microscope is presented which is capable of imaging at excitation wavelengths ranging from (210 nm to 610 nm) with centimeter-scale working distances, sub-micron resolution and operating in reflection mode.

To assess cancer resection margins, post-operative histological diagnosis using hematoxylin and eosin (H&E) stained slides remains the gold standard due to the lack of effective intra-operative approaches. Wait times may be up to two weeks and subsequent treatments may be necessary. Therefore, we are motivated to introduce Chromophore Selective Multi-Wavelength Photoacoustic Remote Sensing, an all-optical, non-contact, reflection-mode, label-free approach to produce H&E-like images of human tissue. This work is a step towards in-situ imaging, rapid clinical assessment of tissue, and may permit future developments as a live intraoperative surgical microscope.