Objective The chest X-ray-relevant screening and diagnostic method is essential for radiology nowadays. Most of chest X-ray images interpretation is still restricted by clinical experience and challenged for misdiagnose and missed diagnoses. To detect and identify one or more potential diseases in images automatically, it is beneficial for improving diagnostic efficiency and accuracy using computer-based technique. Compared to natural images, multiple lesions are challenged to be detected and distinguished accurately in a single image because abnormal areas have a small proportion and complex representations in chest X-ray images. Current convolutional neural network（CNN） based deep learning models have been widely used in the context of medical imaging. The structure of the CNN convolution kernel has sensitive to local detail information, and it is possible to extract richer image features. However, the convolution kernel cannot be used to get global information, and the features-extracted are restricted of redundant information like its relevance of background, muscles, and bones. The model’s performance in multi-label classification tasks are affected to a certain extent. At present, the vision Transformer（ViT） model has achieved its priorities in computer vision-related tasks. The ViT can be used to capture information simultaneously and effectively for multiple regions of the entire image. However, it is required to use large-scale dataset training to achieve good performance. Due to some factors like patient privacy and manual annotate costs, the size of the chest X-ray image data set has been limited. To reduce the model’s dependence on data scale and improve the performance of multi-label classification, we develop the CNN-based ViT pre-training model in terms of the transfer learning method for diagnosis-assisted of chest X-ray image and multi-label classification. Method The CNN-based ViT model is pre-trained on a huge scale ground truth dataset, and it is used to obtain the initial parameters of the model.The model structure is fine-tuned according to the features of chest X-ray dataset. A 1 × 1 convolution layer is used to convert the chest X-ray images channels between 1 to 3. The number of output nodes of the linear layer in the classifier is balanced from 1 000 to the number of chest X-ray classification labels, and the Sigmoid is used as an activation function. The parameters of the backbone network are initialized in terms of the pre-trained ViT model parameters, and it is trained in the chest X-ray dataset after that to complete multi-label classification. The experiment is configured of Python3. 7 and PyTorch1. 8 to construct the model and RTX3090 GPU for training. Stochastic gradient descent（SGD） optimizer, binary cross-entropy（BCE） loss function, an initial learning rate of 1E-3, the cosine annealing learning rate decay are used. For training, each image is scaled to a size of 512 × 512 pixels, and a 224 × 224 pixels area and it is then cropped in random as the model input, and data augmentation is performed randomly by some of the flipping, perspective transformation, shearing, translation, zooming, and changing brightness. For testing, the chest X-ray image is scaled to 256 × 256 pixels and center crop a 224 ×224 area to input the trained model. Result The experiment is performed on the IU X-Ray, which is a small-scale chest X-ray dataset. This model is evaluated in quantitative using the average of area under ROC curve（AUC） scores across all classification labels. The results show that the average AUC score of the pre-trained ViT model is 0. 774. The accuracy and training efficiency of the non-pre-trained ViT model is dropped significantly. The average AUC score is reached to 0. 566 only, which is 0. 208 lower. In addition, the attention mechanism heat map is generated based on the ViT model, which can strengthen the interpretability of the model. A series of ablation experiments are carried out for data augmentation, model structure, and batch size design. The fine-tuned ViT model is trained on the Chest-Ray14and CheXpert dataset as well. The average AUC score is reached to 0. 839 and 0. 806, which is optimized by 0. 014 and 0. 031. Conclusion A pre-trained ViT model is used for the multi-label classification of chest X-ray images via transfer learning. The experimental results illustrate that the ViT has its stronger multi-label classification performance in chest Xray images, and its attention mechanism is beneficial for lesions precision-focused like the interior of the chest cavity and the heart. Transfer learning is potential to improve the classification performance and model generalization of the ViT in small-scale datasets, and the training cost is reduced greatly. Ablation experiments demonstrate that the incorporated model of CNN and Transformer has its priority beyond single-structure model. Data enhancement and the batch size cutting can improve the performance of the model, but smaller scale of batch is still interlinked to longer training span. To improve the model’s ability, we predict that future research direction can be focused on the extraction for complex disease and highlevel semantic information, such as their small lesions, disease location, and severity.

乳腺癌的早期症状在乳腺钼靶图像中主要表现为微钙化点,微钙化区域的真假阳性检测对于乳腺癌早期筛查具有重要意义.首先,对DDSM乳腺数据集中的图像进行预处理,去除噪声及无关组织干扰;其次,基于空-频域差值图像技术实现了疑似微钙化点的分割,取得的敏感性为91.00％,但假阳性率也较高(34.00％),并根据疑似点的质心位置自动截取感兴趣区域;然后,通过超分辨率反馈网络算法进行微钙化区域超分辨率重建;最后,提取感兴趣区域的纹理特征,将Gentle AdaBoost算法和单层决策树算法相结合,构建强分类器GAB-DS对区域进行分类,将微钙化区域和正常组织分离开来,GAB-DS分类模型取得了96.25％的准确率、94.38％的敏感性以及98.13％的特异性.实验结果表明,该模型在微钙化区域检测上性能优越,可用于辅助临床乳腺癌检测及诊断,具有一定的临床应用价值.

为了提高肺结节检测的精确度和效率,提出一种基于多特征融合和XGBoost的肺结节检测模型.首先采用阈值分割与形态学运算,获得候选结节区域;然后通过基于超分辨率重建的卷积神经网络进行候选结节的特征增强;其次采用快速鲁棒特征、灰度共生矩阵、灰度不变矩的提取方法获得候选结节的局部与全局的多种特征,采用词袋模型进行降维并融合;最后利用XGBoost-决策树分类模型去除假阳性结节,完成肺结节的检测.在LIDC-IDRI数据上进行的实验表明该模型能达到97.87％的准确率和97.92％的召回率.该模型可用于辅助医生进行肺结节诊断,具有一定的临床应用价值.