Tuberculosis (TB) is one of the leading infectious causes of death worldwide. The effective management and public health control of this disease depends on early detection and careful treatment monitoring. For many years, the microscopy-based analysis of sputum smears has been the most common method to detect and quantify Mycobacterium tuberculosis (Mtb) bacteria. Nonetheless, this form of analysis is a challenging procedure since sputum examination can only be reliably performed by trained personnel with rigorous quality control systems in place. Additionally, it is affected by subjective judgement. Furthermore, although fluorescence-based sample staining methods have made the procedure easier in recent years, the microscopic examination of sputum is a time-consuming operation. Over the past two decades, attempts have been made to automate this practice. Most approaches have focused on establishing an automated method of diagnosis, while others have centred on measuring the bacterial load or detecting and localising Mtb cells for further research on the phenotypic characteristics of their morphology. The literature has incorporated machine learning (ML) and computer vision approaches as part of the methodology to achieve these goals. In this review, we first gathered publicly available TB sputum smear microscopy image sets and analysed the disparities in these datasets. Thereafter, we analysed the most common evaluation metrics used to assess the efficacy of each method in its particular field. Finally, we generated comprehensive summaries of prior work on ML and deep learning (DL) methods for automated TB detection, including a review of their limitations.

Successful treatment of pulmonary tuberculosis (TB) depends on early diagnosis and careful monitoring of treatment response. Identification of acid-fast bacilli by fluorescence microscopy of sputum smears is a common tool for both tasks. Microscopy-based analysis of the intracellular lipid content and dimensions of individual Mycobacterium tuberculosis (Mtb) cells also describe phenotypic changes which may improve our biological understanding of antibiotic therapy for TB. However, fluorescence microscopy is a challenging, time-consuming and subjective procedure. In this work, we automate examination of fields of view (FOVs) from microscopy images to determine the lipid content and dimensions (length and width) of Mtb cells. We introduce an adapted variation of the UNet model to efficiently localising bacteria within FOVs stained by two fluorescence dyes; auramine O to identify Mtb and LipidTox Red to identify intracellular lipids. Thereafter, we propose a feature extractor in conjunction with feature descriptors to extract a representation into a support vector multi-regressor and estimate the length and width of each bacterium. Using a real-world data corpus from Tanzania, the proposed method i) outperformed previous methods for bacterial detection with a 8% improvement (Dice coefficient) and ii) estimated the cell length and width with a root mean square error of less than 0.01%. Our network can be used to examine phenotypic characteristics of Mtb cells visualised by fluorescence microscopy, improving consistency and time efficiency of this procedure compared to manual methods.

The manual observation of sputum smears by fluorescence microscopy for the diagnosis and treatment monitoring of patients with tuberculosis (TB) is a laborious and subjective task. In this work, we introduce an automatic pipeline which employs a novel deep learning-based approach to rapidly detect Mycobacterium tuberculosis (Mtb) organisms in sputum samples and thus quantify the burden of the disease. Fluorescence microscopy images are used as input in a series of networks, which ultimately produces a final count of present bacteria more quickly and consistently than manual analysis by healthcare workers. The pipeline consists of four stages: annotation by cycle-consistent generative adversarial networks (GANs), extraction of salient image patches, classification of the extracted patches, and finally, regression to yield the final bacteria count. We empirically evaluate the individual stages of the pipeline as well as perform a unified evaluation on previously unseen data that were given ground-truth labels by an experienced microscopist. We show that with no human intervention, the pipeline can provide the bacterial count for a sample of images with an error of less than 5%.

Tuberculosis is one of the most serious infectious diseases, and its treatment is highly dependent on early detection. Microscopy-based analysis of sputum images for bacilli identification is a common technique used for both diagnosis and treatment monitoring. However, it a challenging process since sputum analysis requires time and highly trained experts to avoid potentially fatal mistakes. Capturing fields of view (FOVs) from high resolution whole slide images is a laborious procedure, since they are manually localized and then examined to determine the presence of bacteria. In the present paper we propose a method that automates the process, thus greatly reducing the amount of human labour. In particular, we (i) describe an image processing based method for the extraction of a FOV representation which emphasises salient, bacterial content, while suppressing confounding visual information, and (ii) introduce a novel deep learning based architecture which learns from coarsely labelled FOV images and the corresponding binary masks, and then classifies novel FOV images as salient (bacteria containing) or not. Using a real-world data corpus, the proposed method is shown to outperform 12 state of the art methods in the literature, achieving (i) an approximately 10% lower overall error rate than the next best model and (ii) perfect sensitivity (7% higher than the next best model).

In this paper, our goal is to perform a virtual restoration of an ancient coin from its image. The present work is the first one to propose this problem, and it is motivated by two key promising applications. The first of these emerges from the recently recognised dependence of automatic image based coin type matching on the condition of the imaged coins; the algorithm introduced herein could be used as a pre-processing step, aimed at overcoming the aforementioned weakness. The second application concerns the utility both to professional and hobby numismatists of being able to visualise and study an ancient coin in a state closer to its original (minted) appearance. To address the conceptual problem at hand, we introduce a framework which comprises a deep learning based method using Generative Adversarial Networks, capable of learning the range of appearance variation of different semantic elements artistically depicted on coins, and a complementary algorithm used to collect, correctly label, and prepare for processing a large numbers of images (here 100,000) of ancient coins needed to facilitate the training of the aforementioned learning method. Empirical evaluation performed on a withheld subset of the data demonstrates extremely promising performance of the proposed methodology and shows that our algorithm correctly learns the spectra of appearance variation across different semantic elements, and despite the enormous variability present reconstructs the missing (damaged) detail while matching the surrounding semantic content and artistic style.

Automatic ancient Roman coin analysis only recently emerged as a topic of computer science research. Nevertheless, owing to its ever-increasing popularity, the field is already reaching a certain degree of maturity, as witnessed by a substantial publication output in the last decade. At the same time, it is becoming evident that research progress is being limited by a somewhat veering direction of effort and the lack of a coherent framework which facilitates the acquisition and dissemination of robust, repeatable, and rigorous evidence. Thus, in the present article, we seek to address several associated challenges. To start with, (i) we provide a first overview and discussion of different challenges in the field, some of which have been scarcely investigated to date, and others which have hitherto been unrecognized and unaddressed. Secondly, (ii) we introduce the first data set, carefully curated and collected for the purpose of facilitating methodological evaluation of algorithms and, specifically, the effects of coin preservation grades on the performance of automatic methods. Indeed, until now, only one published work at all recognized the need for this kind of analysis, which, to any numismatist, would be a trivially obvious fact. We also discuss a wide range of considerations which had to be taken into account in collecting this corpus, explain our decisions, and describe its content in detail. Briefly, the data set comprises 100 different coin issues, all with multiple examples in Fine, Very Fine, and Extremely Fine conditions, giving a total of over 650 different specimens. These correspond to 44 issuing authorities and span the time period of approximately 300 years (from 27 BC until 244 AD). In summary, the present article should be an invaluable resource to researchers in the field, and we encourage the community to adopt the collected corpus, freely available for research purposes, as a standard evaluation benchmark.