Deep hashing cross-modal image-text retrieval has the advantage of low storage cost and high retrieval efficiency by mapping different modal data into a Hamming space. However, the existing unsupervised deep hashing methods generally relied on the intrinsic similarity information of each modal for structural matching, failing to fully consider the heterogeneous characteristics and semantic gaps of different modalities, which results in the loss of latent semantic correlation and co-occurrence information between the different modalities. To address this problem, this paper proposes an unsupervised deep hashing with multiple similarity preservation (UMSP) method for cross-modal image-text retrieval. First, to enhance the representation ability of the deep features of each modality, a modality-specific image-text feature extraction module is designed. Specifically, the image network with parallel structure and text network are constructed with the vision-language pre-training image encoder and multi-layer perceptron to capture the deep semantic information of each modality and learn a common hash code representation space. Then, to bridge the heterogeneous gap and improve the discriminability of hash codes, a multiple similarity preservation module is builded based on three perspectives: joint modal space, cross-modal hash space and image modal space, which aids the network to preserve the semantic similarity of modalities. Experimental results on three benchmark datasets (Wikipedia, MIRFlickr-25K and NUS-WIDE) show that UMSP outperforms other unsupervised methods for cross-modal image-text retrieval.

Recently, emotion recognition from facial expressions has achieved unprecedented accuracy with the development of deep learning. Despite this progress, most existing emotion recognition methods are supervised and thus require extensive annotation. This issue is particularly pronounced in continuous domain datasets where annotation costs are very high. Furthermore, discrete domain datasets containing specific poses are too uniform to reflect complex and actual emotions. Existing methods that employ classification loss pay little attention to image similarity, making it difficult to distinguish similar emotions. To improve the learning ability for image similarity and reduce the annotation cost of continuous domain datasets, this research proposes a Semi-Supervised Emotion Recognition (SSER) method, which incorporates Activation-matrix Triplet loss (AMT loss) and pseudo label with Complementary Information (CI label). Specifically, the AMT loss is constructed by encoding multiple activation channels of an image as a matrix, which are utilized to capture the image similarity. The CI label firstly adopts the coupling effect of the complementary information from images and the multi-stage model for SSL to obtain high-confidence pseudo-labels. Then, entropy minimization and consistency regularization are used to improve the accuracy of pseudo labels. The SSER is evaluated on continuous domain datasets (AFEW-VA and AFF-Wild) and discrete domain datasets (FER2013 and CK+). The experimental results demonstrate that the SSER combined with AMT loss and CI label makes improvement for emotion recognition on continuous domain datasets, meanwhile the SSER is also desirable and effective for emotion recognition on discrete domain datasets.

Objective. Skin lesion segmentation plays an important role in the diagnosis and treatment of melanoma. Existing skin lesion segmentation methods have trouble distinguishing hairs, air bubbles, and blood vessels around lesions, which affects the segmentation performance. Approach. To clarify the lesion boundary and raise the accuracy of skin lesion segmentation, a joint attention and adversarial learning network (JAAL-Net) is proposed that consists of a generator and a discriminator. In the JAAL-Net, the generator is a local fusion network (LF-Net) utilizing the encoder-decoder structure. The encoder contains a convolutional block attention module to increase the weight of lesion information. The decoder involves a contour attention to obtain edge information and locate the lesion. To aid the LF-Net generate higher confidence predictions, a discriminant dual attention network is constructed with channel attention and position attention. Main results. The JAAL-Net is evaluated on three datasets ISBI2016, ISBI2017 and ISIC2018. The intersection over union of the JAAL-Net on the three datasets are 90.27%, 89.56% and 80.76%, respectively. Experimental results show that the JAAL-Net obtains rich lesion and boundary information, enhances the confidence of the predictions, and improves the accuracy of skin lesion segmentation. Significance. The proposed approach effectively improves the performance of the model for skin lesion segmentation, which can assist physicians in accurate diagnosis well.

随着深度学习的发展,基于文本生成的隐写术取得了重大突破.现有基于文本生成的方法存在暴露偏差的问题,即训练阶段每个输入都来自真实样本标签,预测阶段的输入来自上一时刻预测的输出.训练和预测之间的输入样本差异会产生误差积累,使得生成样本与真实样本分布相差过大.针对这个问题,提出了一种基于生成对抗网络和多头注意力的文本隐写术—TS-GANMA.首先,利用生成对抗网络训练文本生成器,通过多头注意力机制提取多头注意力得分参与奖惩模块的奖励计算,得到更适合生成器的反馈信息.随后,生成器与鉴别器进行对抗训练,能够解决暴露偏差的问题,优化文本生成模型.最后,对文本生成模型输出的条件概率分布进行编码,实现秘密信息嵌入.实验结果表明,在相同的嵌入率时,TS-GANMA隐写术与 LSTM-vlc 和 ADG 相比,隐写文本的困惑度有显著的降低,这是因为采用TS-GANMA进行文本隐写,生成的隐写文本与真实文本的统计分布更加拟合,生成的隐写文本质量更高.

现有的深度哈希图像检索方法主要采用卷积神经网络，提取的深度特征的相似性表征能力不足.此外，三元组深度哈希主要从小批量数据中构建局部三元组样本，样本数量较少，数据分布缺失全局性，使网络训练不够充分且收敛困难.针对上述问题，文中提出基于类相似特征扩充与中心三元组损失的哈希图像检索模型（Hash Image Retrieval Based on Category Similarity Feature Expansion and Center Triplet Loss, HRFT-Net）.设计基于Vision Transformer的哈希特征提取模块（Hash Feature Extraction Module Based on Vision Transformer, HViT）,利用Vision Transformer提取表征能力更强的全局特征信息.为了扩充小批量训练样本的数据量，提出基于类约束的相似特征扩充模块（Similar Feature Expansion Based on Category Constraint, SFEC）,利用同类样本间的相似性生成新特征，丰富三元组训练样本.为了增强三元组损失的全局性，提出基于Hadamard的中心三元组损失函数（Central Triplet Loss Function Based on Hadamard, CTLH）,利用Hadamard为每个类建立全局哈希中心约束，通过增添局部约束与全局中心约束的中心三元组加速网络的学习和收敛，提高图像检索的精度.在CIFAR10、NUS-WIDE数据集上的实验表明，HRFT-Net在不同长度比特位哈希码检索上的平均精度均值较优，由此验证HRFT-Net的有效性.

为了解决感染区域比正常组织对比度低的问题,提出了一种基于三重注意力机制(triple attention mechanism,TAM)的新冠肺炎(corona virus disease 2019,COVID 19)病灶分割模型——TM-Net,并将其应用于条件生成对抗网络.MultiConv模块可以自动提取肺部切片中感染区域的特征,呈现了更丰富且包含不同类型的病灶信息.TAM同时集成了空间、通道、位置注意力模块,可以更精准地定位感染区域的病灶.使用的损失函数是由3种不同的损失函数组成的复合函数,能最小化预测图和真实标签的差异,更好地优化TM-Net模型.在COVID-19数据集上进行实验和评估的结果表明:基于TM-Net的磨玻璃影(ground-glass opacities,GGO)和固结(Consolidation)两类病灶的平均dice相似系数(dice similarity coefficient,DSC)比基于Attention U-Net和R2U-Net的DSC分别提高了1.4％和0.5％,证明TM-Net提升了新冠肺炎病灶分割的准确性.

在一致性正则化与熵最小化的基础上提出一种新的半监督学习算法Mean Mixup,集成数据的互补信息,然后使用熵最小化给未标记数据生成可靠的伪标签,在一致性正则化下进一步优化模型分类结果.在常用数据集SVHN和CIFAR10上对Mean Mixup算法进行了评估,实验结果表明,所提出的方法在分类准确率上优于一些已有的半监督学习算法.

立足于深度学习,提出面向细粒度图像的自适应三元组网络的鲁棒图像检索算法.首先,提出的视觉显著性检测方法被用来去除图像噪音,以便提取图像中目标主体辨识度更高的深度特征;然后,添加特征增强模块来提高深度特征的表征能力和鲁棒性;最后设计三元组网络,弥补传统分类模型特征判别能力不足的缺陷,获取更适用于细粒度图像检索的网络模型.经实验验证,采用视觉显著性检测、特征增强模块和自适应三元组损失函数方法构建的网络模型提取的深度特征不仅加快检索效率,同时也提高了检索精度.

卷积神经网络因其对图像识别准确率高而在图像检索领域备受青睐,但处理大规模数据集时,基于卷积神经网络提取的深度特征维度高,容易引发"维度灾难".针对图像检索中深度特征维度高的问题,提出一种基于自适应融合网络特征提取与哈希特征降维的图像检索算法.由于传统哈希处理高维特征复杂度高,因此本文在卷积神经网络中加入自适应融合模块对特征进行重新整合,增强特征表征能力的同时降低特征维度;然后应用稀疏化优化算法对深度特征进行第2次降维,并通过映射获得精简的哈希码;最后,实验以Inception网络作为基础模型,在数据集CIFAR-10和ImageNet上进行了丰富的实验.实验结果表明,该算法能有效提高图像检索效率.

Deep convolutional neural networks (DCNNs) have become the dominant machine learning for visual object recognition. They have been widely used in food image recognition and have achieved excellent performance. However, not only are the food-ingredient datasets not easy to obtain, but also the scale is not big enough to learn a deep learning model. For small-scale datasets, this paper proposes a novel DCNN architecture, which constructs an up-to-date combinational convolutional neural network of double subnets (CBDNet) for automatic classification of food ingredients using feature fusion. The feature fusion is a component which aggregates subnets for more abundant and precise deep feature extraction. In order to improve classification accuracy, some useful strategies are adopted, including batch normalisation (BN) operation and hyperparameters setting. Finally, experimental results show that the CBDNet integrating double subnets, feature fusion and BN operation extracts better image features and effectively improves the performance of food-ingredient recognition.

With the development of Deep Convolutional Neural Networks (DCNNs), the extracted features for image recognition tasks have shifted from low-level features to the high-level semantic features of DCNNs. Previous studies have shown that the deeper the network is, the more abstract the features are. However, the recognition ability of deep features would be limited by insufficient training samples. To address this problem, this paper derives an improved Deep Fusion Convolutional Neural Network (DF-Net) which can make full use of the differences and complementarities during network learning and enhance feature expression under the condition of limited datasets. Specifically, DF-Net organizes two identical subnets to extract features from the input image in parallel, and then a well-designed fusion module is introduced to the deep layer of DF-Net to fuse the subnet's features in multi-scale. Thus, the more complex mappings are created and the more abundant and accurate fusion features can be extracted to improve recognition accuracy. Furthermore, a corresponding training strategy is also proposed to speed up the convergence and reduce the computation overhead of network training. Finally, DF-Nets based on the well-known ResNet, DenseNet and MobileNetV2 are evaluated on CIFAR100, Stanford Dogs, and UECFOOD-100. Theoretical analysis and experimental results strongly demonstrate that DF-Net enhances the performance of DCNNs and increases the accuracy of image recognition.

In large-scale image retrieval, deep features extracted by Convolutional Neural Network (CNN) can effectively express more image information than those extracted by traditional manual methods. However, the deep feature dimensions obtained by Deep Convolutional Neural Network (DCNN) are too high and redundant, which leads to low retrieval efficiency. We propose a novel image retrieval method, which combines deep features selection with improved DCNN and hash transform based on high-dimension features reduction to gain lowdimension deep features and realizes efficient image retrieval. Firstly, the improved network is based on the existing deep model to build a more profound and broader network by adding multiple groups of different branches. Therefore, it is named DFS-Net (Deep Feature Selection Network). The adaptive learning deep features of the Network can effectively alleviate the influence of over-fitting and improve the feature expression of image content. Secondly, the information gain rate method is used to filter the extracted deep features to reduce the feature dimension and ensure the information loss is small. The last step of the method, hash Transform, sparsifies and binarizes this representation to reduce the computation and storage pressure while maintaining the retrieval accuracy. Finally, the scheme is based on the distinguished ResNet50, InceptionV3, and MobileNetV2 models, and studied and evaluated deeply on the CIFAR10 and Caltech256 datasets. The experimental results show that the novel method can train the deep features with stronger recognition ability on limited training samples, and improve the accuracy and efficiency of image retrieval effectively.

With the development of deep learning and Convolutional Neural Networks (CNNs), the accuracy of automatic food recognition based on visual data have significantly improved. Some research studies have shown that the deeper the model is, the higher the accuracy is. However, very deep neural networks would be affected by the overfitting problem and also consume huge computing resources. In this paper, a new classification scheme is proposed for automatic food-ingredient recognition based on deep learning. We construct an up-to-date combinational convolutional neural network (CBNet) with a subnet merging technique. Firstly, two different neural networks are utilized for learning interested features. Then, a well-designed feature fusion component aggregates the features from subnetworks, further extracting richer and more precise features for image classification. In order to learn more complementary features, the corresponding fusion strategies are also proposed, including auxiliary classifiers and hyperparameters setting. Finally, CBNet based on the well-known VGGNet, ResNet and DenseNet is evaluated on a dataset including 41 major categories of food ingredients and 100 images for each category. Theoretical analysis and experimental results demonstrate that CBNet achieves promising accuracy for multi-class classification and improves the performance of convolutional neural networks.

With the rapid development of information technology, the speed and efficiency of image retrieval are increasingly required in many fields, and a compelling image retrieval method is critical for the development of information. Feature extraction based on deep learning has become dominant in image retrieval due to their discrimination more complete, information more complementary and higher precision. However, the high-dimension deep features extracted by CNNs (convolutional neural networks) limits the retrieval efficiency and makes it difficult to satisfy the requirements of existing image retrieval. To solving this problem, the high-dimension feature reduction technology is proposed with improved CNN and PCA quadratic dimensionality reduction. Firstly, in the last layer of the classical networks, this study makes a well-designed DR-Module (dimensionality reduction module) to compress the number of channels of the feature map as much as possible, and ensures the amount of information. Secondly, the deep features are compressed again with PCA (Principal Components Analysis), and the compression ratios of the two dimensionality reductions are reduced, respectively. Therefore, the retrieval efficiency is dramatically improved. Finally, it is proved on the Cifar100 and Caltech101 datasets that the novel method not only improves the retrieval accuracy but also enhances the retrieval efficiency. Experimental results strongly demonstrate that the proposed method performs well in small and medium-sized datasets.

Image retrieval for food ingredients is important work, tremendously tiring, uninteresting, and expensive. Computer vision systems have extraordinary advancements in image retrieval with CNNs skills. But it is not feasible for small-size food datasets using convolutional neural networks directly. In this study, a novel image retrieval approach is presented for small and medium-scale food datasets, which both augments images utilizing image transformation techniques to enlarge the size of datasets, and promotes the average accuracy of food recognition with state-of-the-art deep learning technologies. First, typical image transformation techniques are used to augment food images. Then transfer learning technology based on deep learning is applied to extract image features. Finally, a food recognition algorithm is leveraged on extracted deepfeature vectors. The presented image-retrieval architecture is analyzed based on a smallscale food dataset which is composed of forty-one categories of food ingredients and one hundred pictures for each category. Extensive experimental results demonstrate the advantages of image-augmentation architecture for small and medium datasets using deep learning. The novel approach combines image augmentation, ResNet feature vectors, and SMO classification, and shows its superiority for food detection of small/ medium-scale datasets with comprehensive experiments.

目前,卷积神经网络已成为视觉对象识别的主流机器学习方法.有研究表明,网络层数越深,所提取的深度特征表征能力越强.然而,当数据集规模不足时,过深的网络往往容易过拟合,深度特征的分类性能将受到制约.因此,提出了一种新的卷积神经网络分类算法:并行融合网FD-Net.以网络融合的方式提高特征的表达能力,并行融合网首先组织2个相同的子网并行提取图像特征,然后使用精心设计的特征融合器将子网特征进行多尺度融合,提取出更丰富、更精确的融合特征用于分类.此外,采用了随机失活和批量规范化等方法协助特征融合器去除冗余特征,并提出了相应的训练策略控制计算开销.最后,分别以经典的ResNet、InceptionV3、DenseNet和MobileNetV2作为基础模型,在UECFOOD-100和Caltech101等数据集上进行了深入的研究和评估.实验结果表明,并行融合网能在有限的训练样本上训练出识别能力更强的分类模型,有效提高图像的分类准确率.

Test case prioritisation schedules the test cases for execution in an order that attempts to maximise (an) objective(s) or expose faults earlier in testing. In the past, many test case prioritisation techniques prioritised test cases based on mainly test-requirement coverage and ignored many other testing factors. In view of the DU-chain importance in programs, this paper presents a test case prioritisation approach of method-based DU-chain coverage. The technique combines the DU-chain coverage and fault detection rate as test-case quantitative factors. Different from existing techniques, the novel approach makes use of information from executed testing and module coupling, and dynamically calculates a priority quantitative value for every test case. The experiments performed show that the dynamic prioritisation approach is fault-detection effective, and the APFD of the test suites constructed by the dynamic prioritisation approach is higher than that of the test suites constructed by the static prioritisation technique.

The education plays a more and more important role in disseminating knowledge because of the explosive growth of knowledge. As one kind of carrier delivering knowledge, image also presents an explosive growth trend and plays an increasingly important role in education, medical, advertising, entertainment, and so on. Aiming at the long time of massive image feature extraction in the construction of smart campus, the traditional Harris corner has problems, such as low detection efficiency and many non-maximal pseudocorner points. This paper proposes a Harris image matching method that combines adaptive threshold and random sample consensus (RANSAC). First, the Harris feature points are selected based on the adaptive threshold and the Forstner algorithm in this method. On the one hand, candidate points are filtered based on the adaptive threshold. On the other hand, the Forstner algorithm is used to further select the corner points. Second, the normalized cross correlation matching and the RANSAC are applied to precisely match the detected Harris corners. The experimental results show that compared with the existing algorithms, the proposed method not only obtains a matching accuracy higher than 20% of Cui's algorithm but also saves more than 30% detection time of corner detection and image matching. Furthermore, the proposed method obtains a matching accuracy higher than 50% of the Cui's algorithm and saves more than 50% detection time of corner detection and image matching.

Test case prioritization is an effective and practical technique of regression testing.Yet this technique is quite limited in a way that it prioritizes testing cases based on test-requirement coverage only and ignores many other testing factors.To improve the performance,this paper presented a new test case prioritization algorithm based on DU chain.The algorithm combines the DU-chain coverage and fault detection rate as the test-case quantitative factors.Compared with existing algorithms,the new algorithm makes use of information from executed testing and modules coupling,and dynamically calculates priority quantitative value for every test case.The experimental resucts show that the new prioritization algorithm is helpful to detect more faults in a shorter time.