Machine learning techniques have played an essential role in computer-assisted medical image analysisMedical Image Analysis. In this chapter, we will introduce several of our recent achievements with machine learning methods for feature extraction and representation, classification, dense prediction (segmentation and synthesis), and multi-modality analysis, across the pipeline of computer-assisted diagnosis (CAD). These methods consist of both traditional machine learning techniques and state-of-the-art deep learningDeep Learning based approaches. They were proposed to address pain points in the techniques, for example, similarity metric learning for better classification, 3D and sample-adaptive dense prediction models for segmentation and synthesis, and effective multi-modal imaging data fusion. These methods have been employed in different levels of medical imaging applications, such as medical image synthesis within and across imaging modalities, brain tumor segmentation, and mental disease classification. Common approaches used for related research topics are also briefly reviewed.

Iris recognition faces two important issues. they are how to decompose finely and reconstruct the spherical image of the iris, and how to identify the characteristics of the iris. Conventional iris recognition uses usually the planar features of these iris images. However, the human eye is a sphere. The geometric position information of the iris surface is an important signal, but it is difficult to extract the geometric features of the iris sphere from the planar image. Considering the issue that the plane features are prone to distortion and lack fidelity of iris texture, an Orthogonal and Symmetric Spherical Haar Wavelet (OSSHW) basis is proposed to decompose and reconstruct the spherical iris signal to obtain stronger geometric features of iris surface. The comparison of the feature extraction ability to spherical signal by the spherical harmonics and the typical semiorthogonal or nearly orthogonal spherical Haar wavelet is also presented. And then, an iris recognition method based on Convolutional Neural Networks (CNN) + OSSHW is proposed, which can effectively capture the local fine features of iris spherical surface, and has stronger ability in iris recognition than semi-orthogonal or nearly orthogonal spherical Haar wavelet bases.

Dose reduction in cerebral CT perfusion (CTP) imaging is desirable but is accompanied by an increase in noise that can compromise the image quality and the accuracy of image-based haemodynamic modelling used for clinical decision support in acute ischaemic stroke. The few reported methods aimed at denoising low-dose CTP images lack practicality by considering only small sections of the brain or being computationally expensive. Moreover, the prediction of infarct and penumbra size and location-the chief means of decision support for treatment options-from denoised data has not been explored using these approaches. In this work, we present the first application of a 3D generative adversarial network (3D GAN) for predicting normal-dose CTP data from low-dose CTP data. Feasibility of the approach was tested using real data from 30 acute ischaemic stroke patients in conjunction with low dose simulation. The 3D GAN model was applied to 643voxel patches extracted from two different configurations of the CTP data-frame-based and stacked. The method led to whole-brain denoised data being generated for haemodynamic modelling within 90 s. Accuracy of the method was evaluated using standard image quality metrics and the extent to which the clinical content and lesion characteristics of the denoised CTP data were preserved. Results showed an average improvement of 5.15-5.32 dB PSNR and 0.025-0.033 structural similarity index (SSIM) for CTP images and 2.66-3.95 dB PSNR and 0.036-0.067 SSIM for functional maps at 50% and 25% of normal dose using GAN model in conjunction with a stacked data regime for image synthesis. Consequently, the average lesion volumetric error reduced significantly (p-value <0.05) by 18%-29% and dice coefficient improved significantly by 15%-22%. We conclude that GAN-based denoising is a promising practical approach for reducing radiation dose in CTP studies and improving lesion characterisation.

Medical images have been widely used in clinics, providing visual representations of under-skin tissues in human body. By applying different imaging protocols, diverse modalities of medical images with unique characteristics of visualization can be produced. Considering the cost of scanning high-quality single modality images or homogeneous multiple modalities of images, medical image synthesis methods have been extensively explored for clinical applications. Among them, deep learning approaches, especially convolutional neural networks (CNNs) and generative adversarial networks (GANs), have rapidly become dominating for medical image synthesis in recent years. In this chapter, based on a general review of the medical image synthesis methods, we will focus on introducing typical CNNs and GANs models for medical image synthesis. Especially, we will elaborate our recent work about low-dose to high-dose PET image synthesis, and cross-modality MR image synthesis, using these models.

Generative adversarial network (GAN) has been widely explored for cross-modality medical image synthesis. The existing GAN models usually adversarially learn a global sample space mapping from the source-modality to the target-modality and then indiscriminately apply this mapping to all samples in the whole space for prediction. However, due to the scarcity of training samples in contrast to the complicated nature of medical image synthesis, learning a single global sample space mapping that is "optimal" to all samples is very challenging, if not intractable. To address this issue, this paper proposes sample-adaptive GAN models, which not only cater for the global sample space mapping between the source- and the target-modalities but also explore the local space around each given sample to extract its unique characteristic. Specifically, the proposed sample-adaptive GANs decompose the entire learning model into two cooperative paths. The baseline path learns a common GAN model by fitting all the training samples as usual for the global sample space mapping. The new sample-adaptive path additionally models each sample by learning its relationship with its neighboring training samples and using the target-modality features of these training samples as auxiliary information for synthesis. Enhanced by this sample-adaptive path, the proposed sample-adaptive GANs are able to flexibly adjust themselves to different samples, and therefore optimize the synthesis performance. Our models have been verified on three cross-modality MR image synthesis tasks from two public datasets, and they significantly outperform the state-of-the-art methods in comparison. Moreover, the experiment also indicates that our sample-adaptive strategy could be utilized to improve various backbone GAN models. It complements the existing GANs models and can be readily integrated when needed.

Intensity variation among MR images increases the difficulty of training a segmentation model and generalizing it to unseen MR images. To solve this problem, we propose to learn a sample-adaptive intensity lookup table (LuT) that adjusts each image’s contrast dynamically so that the resulting images could better serve the subsequent segmentation task. Specifically, our proposed deep SA-LuT-Net consists of an LuT module and a segmentation module, trained in an end-to-end manner: the LuT module learns a sample-specific piece-wise linear intensity mapping function under the guide of the performance of the segmentation module. We develop our SA-LuT-Nets based on two backbone networks, DMFNet and the modified 3D Unet, respectively, and validate them on BRATS2018 dataset for brain tumor segmentation. Our experiment results clearly show the effectiveness of SA-LuT-Net in the scenarios of both single and multi-modalities, which is superior over the two baselines and many other relevant state-of-the-art segmentation models.

Magnetic resonance (MR) imaging is a widely used medical imaging protocol that can be configured to provide different contrasts between the tissues in human body. By setting different scanning parameters, each MR imaging modality reflects the unique visual characteristic of scanned body part, benefiting the subsequent analysis from multiple perspectives. To utilize the complementary information from multiple imaging modalities, cross-modality MR image synthesis has aroused increasing research interest recently. However, most existing methods only focus on minimizing pixel/voxel-wise intensity difference but ignore the textural details of image content structure, which affects the quality of synthesized images. In this paper, we propose edge-aware generative adversarial networks (Ea-GANs) for cross-modality MR image synthesis. Specifically, we integrate edge information, which reflects the textural structure of image content and depicts the boundaries of different objects in images, to reduce this gap. Corresponding to different learning strategies, two frameworks are proposed, i.e., a generator-induced Ea-GAN (gEa-GAN) and a discriminator-induced Ea-GAN (dEa-GAN). The gEa-GAN incorporates the edge information via its generator, while the dEa-GAN further does this from both the generator and the discriminator so that the edge similarity is also adversarially learned. In addition, the proposed Ea-GANs are 3D-based and utilize hierarchical features to capture contextual information. The experimental results demonstrate that the proposed Ea-GANs, especially the dEa-GAN, outperform multiple state-of-the-art methods for cross-modality MR image synthesis in both qualitative and quantitative measures. Moreover, the dEa-GAN also shows excellent generality to generic image synthesis tasks on benchmark datasets about facades, maps, and cityscapes.

CT Perfusion (CTP) imaging is one of the most common regimes for evaluation of acute ischemic stroke patients. The CTP imaging protocol typically involves the rapid acquisition of several frames of the brain volume over ~1 minute, following contrast administration. Therefore, it is associated with a relatively high radiation dose. The ability to reduce this dose, while maintaining the accuracy of image-based stroke analysis is highly desirable. However, a reduction in dose is accompanied by an increase in noise, which can compromise computation of important haemodynamic parameters during stroke analysis. In this paper, we investigate the feasibility of using 3D conditional generative adversarial networks (3D c-GANs) to achieve CTP dose reduction while preserving image quality. We simulated low-dose CTP images corresponding to tube currents of 100 mAs and 45 mAs for 18 positive acute stroke subjects and applied a 3D c-GANs model to estimate the standard dose CTP images from the simulated low-dose images. We also compared two different strategies for handling the 4D nature of the CTP data in the 3D c-GANs model. Qualitatively, the results showed excellent agreement between the estimated low-noise images and the true images. Quantitative assessment also showed good performance of the model associated with high peak signal-to-noise ratio (PNSR) around 40 dB, normalized mean squared error (NMSE) close to zero, and structural similarity index (SSIM) close to 1. By stacking the original data rather than concatenating all volumes, the results were improved by 1.05 dB PNSR, 0.005 NMSE, and 0.01 SSIM at the simulated exposure of 100mAs, and by 0.93dB PNSR, 0.019 NMSE at the simulated tube current of 45 mAs. The results show good promise for dose reduction in CTP, however we are currently performing full stroke modelling analysis on the synthetic images to validate the method.

Positron emission topography (PET) has been substantially used in recent years. To minimize the potential health risks caused by the tracer radiation inherent to PET scans, it is of great interest to synthesize the high-quality full-dose PET image from the low-dose one to reduce the radiation exposure while maintaining the image quality. In this paper, we propose a locality adaptive multi-modality generative adversarial networks model (LA-GANs) to synthesize the full-dose PET image from both the low-dose one and the accompanying T1-weighted MRI to incorporate anatomical information for better PET image synthesis. This paper has the following contributions. First, we propose a new mechanism to fuse multi-modality information in deep neural networks. Different from the traditional methods that treat each image modality as an input channel and apply the same kernel to convolute the whole image, we argue that the contributions of different modalities could vary at different image locations, and therefore a unified kernel for a whole image is not appropriate. To address this issue, we propose a method that is locality adaptive for multi-modality fusion. Second, to learn this locality adaptive fusion, we utilize 1 × 1 × 1 kernel so that the number of additional parameters incurred by our method is kept minimum. This also naturally produces a fused image which acts as a pseudo input for the subsequent learning stages. Third, the proposed locality adaptive fusion mechanism is learned jointly with the PET image synthesis in an end-to-end trained 3D conditional GANs model developed by us. Our 3D GANs model generates high quality PET images by employing large-sized image patches and hierarchical features. Experimental results show that our method outperforms the traditional multi-modality fusion methods used in deep networks, as well as the state-of-the-art PET estimation approaches.

Positron emission tomography (PET) is a widely used imaging modality, providing insight into both the biochemical and physiological processes of human body. Usually, a full dose radioactive tracer is required to obtain high-quality PET images for clinical needs. This inevitably raises concerns about potential health hazards. On the other hand, dose reduction may cause the increased noise in the reconstructed PET images, which impacts the image quality to a certain extent. In this paper, in order to reduce the radiation exposure while maintaining the high quality of PET images, we propose a novel method based on 3D conditional generative adversarial networks (3D c-GANs) to estimate the high-quality full-dose PET images from low-dose ones. Generative adversarial networks (GANs) include a generator network and a discriminator network which are trained simultaneously with the goal of one beating the other. Similar to GANs, in the proposed 3D c-GANs, we condition the model on an input low-dose PET image and generate a corresponding output full-dose PET image. Specifically, to render the same underlying information between the low-dose and full-dose PET images, a 3D U-net-like deep architecture which can combine hierarchical features by using skip connection is designed as the generator network to synthesize the full-dose image. In order to guarantee the synthesized PET image to be close to the real one, we take into account of the estimation error loss in addition to the discriminator feedback to train the generator network. Furthermore, a concatenated 3D c-GANs based progressive refinement scheme is also proposed to further improve the quality of estimated images. Validation was done on a real human brain dataset including both the normal subjects and the subjects diagnosed as mild cognitive impairment (MCI). Experimental results show that our proposed 3D c-GANs method outperforms the benchmark methods and achieves much better performance than the state-of-the-art methods in both qualitative and quantitative measures.

Different modalities of magnetic resonance imaging (MRI) can indicate tumor-induced tissue changes from different perspectives, thus benefit brain tumor segmentation when they are considered together. Meanwhile, it is always interesting to examine the diagnosis potential from single modality, considering the cost of acquiring multi-modality images. Clinically, T1-weighted MRI is the most commonly used MR imaging modality, although it may not be the best option for contouring brain tumor. In this paper, we investigate whether synthesizing FLAIR images from T1 could help improve brain tumor segmentation from the single modality of T1. This is achieved by designing a 3D conditional Generative Adversarial Network (cGAN) for FLAIR image synthesis and a local adaptive fusion method to better depict the details of the synthesized FLAIR images. The proposed method can effectively handle the segmentation task of brain tumors that vary in appearance, size and location across samples.

Accurate segmentation of head-and-neck tumor from medical images is crucial for diagnosis and treatment in clinical field. Compared with other types of tumor, the nasopharyngeal carcinoma (NPC) tumor has more complex anatomical structure and often shares similar imaging intensities with the nearby tissues such as brainstem, parotid and lymph, making the segmentation of NPC tumor particularly difficult. In this paper, to take advantage of multi-modality medical information, we propose a multi-modality joint dictionary learning method for NPC tumor segmentation. The tumor segmentation task is formulated as a voxel-wise labeling problem with regard to two classes: NPC tumor and normal tissues. In our method, both the multi-modality samples with CT and MRI images as well as the single-modality samples with only CT or MRI images are effectively utilized to perform joint dictionary learning. Experimental results show that our proposed method outperforms the benchmark method and achieves comparable results with prior NPC segmentation methods.

Positron emission tomography (PET) has been substantially used recently. To minimize the potential health risk caused by the tracer radiation inherent to PET scans, it is of great interest to synthesize the high-quality PET image from the low-dose one to reduce the radiation exposure. In this paper, we propose a 3D auto-context-based locality adaptive multi-modality generative adversarial networks model (LA-GANs) to synthesize the high-quality FDG PET image from the low-dose one with the accompanying MRI images that provide anatomical information. Our work has four contributions. First, different from the traditional methods that treat each image modality as an input channel and apply the same kernel to convolve the whole image, we argue that the contributions of different modalities could vary at different image locations, and therefore a unified kernel for a whole image is not optimal. To address this issue, we propose a locality adaptive strategy for multi-modality fusion. Second, we utilize 1 ×1 ×1 kernel to learn this locality adaptive fusion so that the number of additional parameters incurred by our method is kept minimum. Third, the proposed locality adaptive fusion mechanism is learned jointly with the PET image synthesis in a 3D conditional GANs model, which generates high-quality PET images by employing large-sized image patches and hierarchical features. Fourth, we apply the auto-context strategy to our scheme and propose an auto-context LA-GANs model to further refine the quality of synthesized images. Experimental results show that our method outperforms the traditional multi-modality fusion methods used in deep networks, as well as the state-of-the-art PET estimation approaches.

Data clustering is a meaningful tool that can, help people classify mixed data automatically. With rapid technological development, data in modern applications become large scale and high dimensional. Some original clustering methods are not suitable for complicated datasets. To improve the performance of the popular kernel fuzzy C-means (KFCM), this study proposed a local density adaptive diffusion maps (LDM) technique to obtain a reliable similarity description and dimensionality reduction. To find the valid cluster centroids of the dataset, this study also proposed an improved cuckoo search (ICS) to optimize the unknown parameters of the KFCM model. The ICS algorithm utilized quaternions to represent individuals who will be optimized. Variable step length of Lévy flights and discovery probability were also proposed, which were adjusted by the evolutional ratio of the cuckoo search process. To verify the availability of the ICS, 5 benchmark functions were tested. Finally, the proposed hybrid ICS and LDM based on KFCM (ICS-LDM-KFCM) was used to identify 4 standard artificial and 6 real world datasets. Compared with other clustering methods, the proposed method obtained more accurate results. This method is verified to be more suitable for complicated datasets with large number of attributes and clusters.

This study proposed a new method for multi-focus image fusion using hybrid wavelet and classifier. The image fusion process was formulated as a two-class classification problem: in and out-of-focus classes. First, a six-dimensional feature vector was extracted using sub-bands of dual-tree complex wavelet transform (DT-CWT) coefficients from the source images, which were then projected by a trained two-class support vector machine (SVM) to the class labels. A bacterial foraging optimization algorithm (BFOA) was developed to obtain the optimal parameters of the SVM. The output of the classification system was used as a decision matrix for fusing high-frequency wavelet coefficients from multi-focus source images in different directions and decomposition levels of the DT-CWT. After the high and low-frequency coefficients of the source images were fused, the final fused image was obtained using the inverse DT-CWT. Several existing methods were compared with the proposed method. Experimental results showed that our presented method outperformed the existing methods, in visual effect and in objective evaluation.

In this study, a novel BF–PSO–FSVCM model has been proposed to identify the fatigue status of the electromyography (EMG) signal. To improve the classifier accuracy of fuzzy support vector classification machine (FSVCM), a hybrid Bacterial Foraging (BF) and particle swarm optimization (PSO) is proposed to optimize the unknown parameters of the classifier. In the proposed method, the EMG signals are firstly decomposed by discrete wavelet transform (DWT), Fast Fourier Transformation (FFT) and Ensemble Empirical Mode Decomposition (EEMD)–Hilbert transform (HT), and then a set of combined features were extracted from different types of fatigue or normal EMG signals. The optimal fatigue vectors of static, local and dynamic fatigue are also provided in this study. The obtained results obviously indicate that further significant enhancements in terms of classification accuracy can be achieved by the proposed BF–PSO–FSVCM classification system. BF–PSO–FSVCM is developed as an efficient tool so that various support vector classification machines (SVCMs) can be used conveniently as the core of BF–PSO–FSVCM for diagnosis of fatigue status.

Cognitive state, which is the inner mental state of a person while interacting with an artificial system through man-machine interface, can be affected by various factors, such as fatigue, stress, mental workload, attention deficit, and executive function, among others, which can lead to errors, accidents, or even disasters. One practical solution to this problem is to monitor and recognize the cognitive state of subjects via physiological signals. In this study, a hybrid adaptive flower pollination algorithm-Gaussian process model is proposed to recognize the cognitive state of in-flight pilots. Instead of using the traditional conjugate gradient technique to find optimal hyperparameters, an improved flower pollination algorithm is proposed. The adaptive Lévy strategy is then used to increase the robustness of this algorithm, as well as to enhance the global optimization and generalization capability of the Gaussian process model. In addition to conventional features in the time-frequency domain, a novel set of features involving wavelet singular entropy and autoregressive-moving average entropy is proposed to improve classification accuracy. Experiments are performed through flight simulations in a full flight simulator with six degrees of freedom. Comparable experimental results validate the feasibility of the proposed method for recognizing cognitive state and provide a wide range of conclusions on the feature selection and feature patterns of cognitive state.

Affinity propagation (AP) is an efficient clustering technique to deal with datasets of many instances; however, it has oscillations and its preference value needs to be preset. This study proposes an improved cuckoo search (ICS) technique to solve the AP model. The ICS algorithm utilizes quaternions to represent individuals that are to be optimized. The variable step length of Levy flights and a method of discovering probability are also proposed. The proposed adaptive AP based on ICS is utilized (or tested) to identify four standard test datasets, such as face images and handwritten digits. The proposed method produces highly accurate results. (C) 2016 Elsevier B.V. All rights reserved.

This paper proposed a novel data reduction and classification method to analyze high-dimensional and complicated flight data. This method integrated diffusion maps and kernel fuzzy c-means algorithm (KFCM) to recognize two types of simulator modes at different tasks. To optimize the unknown parameters of the KFCM, a hybrid bacterial foraging oriented (BFO) and particle swarm optimization (PSO) algorithm was also presented in this paper. This algorithm increased the possibility of finding the optimal values within a short computational time and avoided to be trapped in the local minima. By using the proposed approach, this paper obtained meaningful clusters respecting the intrinsic geometry of the standard data set, and illustrated the phenomenon that the pilots vestibular influenced pilot performance and control system under the Manual departure task.