While buying a product from the e-commerce websites, customers generally have a plethora of questions. From the perspective of both the e-commerce service provider as well as the customers, there must be an effective question answering system to provide immediate answers to the user queries. While certain questions can only be answered after using the product, there are many questions which can be answered from the product specification itself. Our work takes a first step in this direction by finding out the relevant product specifications, that can help answering the user questions. We propose an approach to automatically create a training dataset for this problem. We utilize recently proposed XLNet and BERT architectures for this problem and find that they provide much better performance than the Siamese model, previously applied for this problem. Our model gives a good performance even when trained on one vertical and tested across different verticals.

In this work we present a fast approach to perform registration of computed tomography to ultrasound volumes for image guided intervention applications. The method is based on a combination of block matching and outlier rejection. The block-matching uses a correlation based multimodal similarity metric, where the intensity and the gradient of the computed tomography images along with the ultrasound volumes are the input images to find correspondences between blocks in the computed tomography and the ultrasound volumes. A variance and octree based feature point-set selection method is used for selecting distinct and evenly spread point locations for block-matching. Geometric consistency and smoothness criteria are imposed in an outlier rejection step to refine the block-matching results. The block-matching results after outlier rejection are used to determine the affine transformation between the computed tomography and the ultrasound volumes. Various experiments are carried out to assess the optimal performance and the influence of parameters on accuracy and computational time of the registration. A leave-one-patient-out cross-validation registration error of 3.6 mm is achieved over 29 datasets, acquired from 17 patients. (C) 2019 Elsevier B.V. All rights reserved.

PURPOSE:Compensation for respiratory motion is important during abdominal cancer treatments. In this work we report the results of the 2015 MICCAI Challenge on Liver Ultrasound Tracking and extend the 2D results to relate them to clinical relevance in form of reducing treatment margins and hence sparing healthy tissues, while maintaining full duty cycle. METHODS:We describe methodologies for estimating and temporally predicting respiratory liver motion from continuous ultrasound imaging, used during ultrasound-guided radiation therapy. Furthermore, we investigated the trade-off between tracking accuracy and runtime in combination with temporal prediction strategies and their impact on treatment margins. RESULTS:Based on 2D ultrasound sequences from 39 volunteers, a mean tracking accuracy of 0.9 mm was achieved when combining the results from the 4 challenge submissions (1.2 to 3.3 mm). The two submissions for the 3D sequences from 14 volunteers provided mean accuracies of 1.7 and 1.8 mm. In combination with temporal prediction, using the faster (41 vs 228 ms) but less accurate (1.4 vs 0.9 mm) tracking method resulted in substantially reduced treatment margins (70% vs 39%) in contrast to mid-ventilation margins, as it avoided non-linear temporal prediction by keeping the treatment system latency low (150 vs 400 ms). Acceleration of the best tracking method would improve the margin reduction to 75%. CONCLUSIONS:Liver motion estimation and prediction during free-breathing from 2D ultrasound images can substantially reduce the in-plane motion uncertainty and hence treatment margins. Employing an accurate tracking method while avoiding non-linear temporal prediction would be favorable. This approach has the potential to shorten treatment time compared to breath-hold and gated approaches, and increase treatment efficiency and safety.

We propose a spatial compounding technique and variational framework to improve 3D ultrasound image quality by compositing multiple ultrasound volumes acquired from different probe orientations. In the composite volume, instead of intensity values, we estimate a tensor at every voxel. The resultant tensor image encapsulates the directional information of the underlying imaging data and can be used to generate ultrasound volumes from arbitrary, potentially unseen, probe positions. Extending the work of Hennersperger et al.,1 we introduce a log-Euclidean framework to ensure that the tensors are positive-definite, eventually ensuring non-negative images. Additionally, we regularise the underpinning ill-posed variational problem while preserving edge information by relying on a total variation penalisation of the tensor field in the log domain. We present results on in vivo human data to show the efficacy of the approach.

Automatic vessel extraction from X-ray angiograms (XA) for percutaneous coronary interventions is often hampered by low contrast and presence of background structures, e.g. diaphragm, guiding catheters, stitches. In this paper, we present a novel layer separation technique for vessel enhancement in XA to address this problem. The method uses morphological filtering and Robust PCA to separate interventional XA images into three layers, i.e. a large-scale breathing structure layer, a quasi-static background layer and a layer containing the vessel structures that could potentially improve the quality of vessel extraction from XA. The method is evaluated on several clinical XA sequences. The result shows that the proposed method significantly increases the visibility of vessels in XA and outperforms other background-removal methods.

In this work we describe a 4D registration method for on the fly stabilization of ultrasound volumes for improving image guidance for transjugular intrahepatic portosystemic shunt (TIPS) interventions. The purpose of the method is to enable a continuous visualization of the relevant anatomical planes (determined in a planning stage) in a free breathing patient during the intervention. This requires registration of the planning information to the interventional images, which is achieved in two steps. In the first step tracking is performed across the streaming input. An approximate transformation between the reference image and the incoming image is estimated by composing the intermediate transformations obtained from the tracking. In the second step a subsequent registration is performed between the reference image and the approximately transformed incoming image to account for the accumulation of error. The two step approach helps in reducing the search range and is robust under rotation. We additionally present an approach to initialize and verify the registration. Verification is required when the reference image (containing planning information) is acquired in the past and is not part of the (interventional) 4D ultrasound sequence. The verification score will help in invalidating the registration outcome, for instance, in the case of insufficient overlap or information between the registering images due to probe motion or loss of contact, respectively. We evaluate the method over thirteen 4D US sequences acquired from eight subjects. A graphics processing unit implementation runs the 4D tracking at 9 Hz with a mean registration error of 1.7 mm.

In this paper we present a rigid registration approach for 4D ultrasound (US) datasets, where images are registered over time. The 3D registration approach preceding the 4D registration consists of two main steps -block-matching and outlier rejection. The outlier rejection step removes the spurious matchings' from the block-matching module and ensures inverse consistency. For 4D registration, we perform registration of consecutive US volumes over the time series. Transformation between any two frames is estimated by taking the product of all the intermediate transforms. To avoid accumulation of error over the series of transformations, a long range feedback mechanism is proposed. A mean total registration error of 1 mm is achieved across six 4D ultrasound sequences of human liver with an execution speed of 10 Hz.

Real-time 3D US has potential for image guidance in minimally invasive liver interventions. However, motion caused by patient breathing makes it hard to visualize a localized area, and to maintain alignment with pre-operative information. In this work we develop a fast affine registration framework to compensate in real-time for liver motion/displacement due to breathing. The affine registration of two consecutive ultrasound volumes in time is performed using block-matching. For a set of evenly distributed points in one volume and their correspondences in the other volume, we propose a robust outlier rejection method to reject false matches. The inliers are then used to determine the affine transformation. The approach is evaluated on 13 4D ultrasound sequences acquired from 8 subjects. For 91 pairs of 3D ultrasound volumes selected from these sequences, a mean registration error of 1.8mm is achieved. A graphics processing unit (GPU) implementation runs the 3D US registration at 8 Hz.

Local binary patterns [LBP][1] are popular texture descriptors in many image analysis tasks. One of the important aspects of this texture descriptor is their rotational invariance. Most work in LBP has focused on 2D images. Here, we present a three dimensional LBP with a rotational invariant operator using spherical harmonics. Unlike Fehr and Burkhardt [2], the invariance is constructed implicitly, without considering all possible combinations of the pattern. We demonstrate the 3D LBP on phantom data and a clinical CTA dataset.

Acquisition of a clinically acceptable scan plane is a pre-requisite for ultrasonic measurement of anatomical features from B-mode images. In obstetric ultrasound, measurement of gestational age predictors, such as biparietal diameter and head circumference, is performed at the level of the thalami and cavum septum pelucidi. In an accurate scan plane, the head can be modeled as an ellipse, the thalami looks like a butterfly, the cavum appears like an empty box and the falx is a straight line along the major axis of a symmetric ellipse inclined either parallel to or at small angles to the probe surface. Arriving at the correct probe placement on the mother's belly to obtain an accurate scan plane is a task of considerable challenge especially for a new user of ultrasound. In this work, we present a novel automated learning-based algorithm to identify an acceptable fetal head scan plane. We divide the problem into cranium detection and a template matching to capture the composite "butterfly" structure present inside the head, which mimics the visual cues used by an expert. The algorithm uses the stateof- the-art Active Appearance Models techniques from the image processing and computer vision literature and tie them to presence or absence of the inclusions within the head to automatically compute a score to represent the goodness of a scan plane. This automated technique can be potentially used to train and aid new users of ultrasound.

Fetal bi-parietal diameter (BPD) is known to provide a reliable estimate of gestational age (GA) of a fetus in the first half of pregnancy. In this paper, we present an automated method to identify and measure BPD from B-mode ultrasound images of fetal head. The method (a) automatically detects and places a region-of-interest on the head based on a prior work in our group (b) utilizes the concept of phase congruency for edge detection and (c) employs a cost function to identify the third ventricle inside the head (d) measures the BPD along the perpendicular bisector of occipital frontal diameter (OFD) from the outer rim of the cranium closer to the transducer to the inner rim of the cranium away from the transducer. The cost function is premised on the distribution of anatomical shape, size and presentation of the third ventricle in images that adhere to clinical guidelines describing the scan plane for BPD measurement. The OFD is assumed to lie along the third ventricle. The algorithm has been tested on 137 images acquired from four different scanners. Based on GA estimates and their bounds specified in Standard Obstetric Tables, the GA predictions from automated measurements are found to be within +/- 2SD of GA estimates from manual measurements by the operator and a second expert radiologist in 98% of the cases. The method described in this paper can also be adapted to assess the accuracy of the scan plane based on the presence/absence of the third ventricle.

Traditional methods of detection tend to under perform in the presence of the strong and variable background clutter that characterize a medical ultrasound image. In this paper, we present a novel diffusion based technique to localize acoustically dense objects in an ultrasound image. The approach is premised on the observation that the topology of noise in ultrasound images is more sensitive to diffusion than that of any such physical object (≫ λ). We show that our method when applied to the problem of fetal head detection and automatic measurement of head circumference in 59 obstetric scans compares remarkably well with manually assisted measurements. Based on fetal age estimates and their bounds specified in Standard OB Tables, the Gestational Age predictions from automated measurements is found to be within ± 2SD in 95% and 98% of cases when compared with manual measurements by two experts. The framework is general and can be extended to object localization in diverse applications of ultrasound imaging.

We propose an approach to restore severely degraded document images using a probabilistic context model. Unlike traditional approaches that use previously learned prior models to restore an image, we are able to learn the text model from the degraded document itself, making the approach independent of script, font, style, etc. We model the contextual relationship using an MRF. The ability to work with larger patch sizes allows us to deal with severe degradations including cuts, blobs, merges and vandalized documents. Our approach can also integrate document restoration and super-resolution into a single framework, thus directly generating high quality images from degraded documents. Experimental results show significant improvement in image quality on document images collected from various sources including magazines and books, and comprehensively demonstrate the robustness and adaptability of the approach. It works well with document collections such as books, even with severe degradations, and hence is ideally suited for repositories such as digital libraries.