Sampling a limited number of phase-encoding lines followed by estimating missing lines is an efficient method for shortening scan time of MRI. GeneRalized Autocalibarating Partial Parallel Acquisition (GRAPPA) is such a classical method and is widely used in clinical MRI. As a non-linear method, Robust Artificial-neural-networks for K-space Interpolation (RAKI) is a break-through of GRAPPA in the sense of much higher estimation accuracy. However, RAKI takes much longer estimation time because it requires online training a network for each receiving coil. To overcome the low-efficiency problem, we propose a fast version of RAKI (called fRAKI). fRAKI is roughly 26 times faster and can obtain much higher estimation accuracy compared with RAKI. The high efficiency of fRAKI is due to two properties: (1) A single network is shared to estimate missing lines of all the coils. (2) The online training of fRAKI can converge after a smaller number of iterations. Fast convergency is obtained by using a pre-trained model for initializing learnable parameters. High accuracy benefits from that the pre-train model contains data-universal prior and is also used as a sub-network of fRAKI so that the online training subnetwork can focus on learning scan-specific prior without the risk of overfitting the scan-specific data. Experimental results on the NYU fastMRI knee and brain datasets demonstrate the efficiency and accuracy of the proposed fRAKI.

Stable Diffusion has shown strong ability to generate high-quality and diverse images. However, Stable Diffusion suffers from high computational cost, due to the heavy model and step-by-step denoising process. To address these issues, we propose a token-aware and step-aware acceleration approach for Stable Diffusion, named TSA-SD. We first build a simple and efficient baseline by combining exiting intra-step and cross-step acceleration strategies, including token merging and feature caching, into Stable Diffusion. To improve image generation quality of the baseline, we introduce token-aware merging-unmerging and step-aware acceleration. The token-aware merging-unmerging aims to select informative tokens when merging and recover merged tokens using token ratio information. Therefore, the token-aware merging-unmerging can fully utilize token- specific information, thereby reducing token information loss. In addition, we observe that different steps have different functional linearity, and propose step-aware acceleration to perform different merging operations according to functional linearity at different steps. With these two modules, our proposed TSA-SD is able to generate high-quality images at a high speed. We perform the experiments on two widely-used datasets, including ImageNet and MS-COCO. The experimental results demonstrate the effectiveness and efficiency of our proposed method. For instance, on ImageNet validation set, compared to Stable Diffusion, ToMe-SD has a lower FID of 33.68 at 1.96x speedup, while our method achieves a lower FID of 32.49 at 4.68x speedup.

In the medical field, the application of machine learning technology in the automatic diagnosis and monitoring of osteoporosis often faces challenges related to domain adaptation in drug therapy research. The existing neural networks used for the diagnosis of osteoporosis may experience a decrease in model performance when applied to new data domains due to changes in radiation dose and equipment. To address this issue, in this study, we propose a new method for multi domain diagnostic and quantitative computed tomography (QCT) images, called DeepmdQCT. This method adopts a domain invariant feature strategy and integrates a comprehensive attention mechanism to guide the fusion of global and local features, effectively improving the diagnostic performance of multi domain CT images. We conducted experimental evaluations on a self-created OQCT dataset, and the results showed that for dose domain images, the average accuracy reached 91%, while for device domain images, the accuracy reached 90.5%. our method successfully estimated bone density values, with a fit of 0.95 to the gold standard. Our method not only achieved high accuracy in CT images in the dose and equipment fields, but also successfully estimated key bone density values, which is crucial for evaluating the effectiveness of osteoporosis drug treatment. In addition, we validated the effectiveness of our architecture in feature extraction using three publicly available datasets. We also encourage the application of the DeepmdQCT method to a wider range of medical image analysis fields to improve the performance of multi-domain images.

A multi-feature fusion DCNN model for automated evaluation of lumbar vertebrae L1 on chest combined with clinical information and radiomics permits estimation of volumetric bone mineral density for evaluation of osteoporosis. To develop a multi-feature deep learning model based on chest CT, combined with clinical information and radiomics to explore the feasibility in screening for osteoporosis based on estimation of volumetric bone mineral density. The chest CT images of 1048 health check subjects were retrospectively collected as the master dataset, and the images of 637 subjects obtained from a different CT scanner were used for the external validation cohort. The subjects were divided into three categories according to the quantitative CT (QCT) examination, namely, normal group, osteopenia group, and osteoporosis group. Firstly, a deep learning–based segmentation model was constructed. Then, classification models were established and selected, and then, an optimal model to build bone density value prediction regression model was chosen. The DSC value was 0.951 ± 0.030 in the testing dataset and 0.947 ± 0.060 in the external validation cohort. The multi-feature fusion model based on the lumbar 1 vertebra had the best performance in the diagnosis. The area under the curve (AUC) of diagnosing normal, osteopenia, and osteoporosis was 0.992, 0.973, and 0.989. The mean absolute errors (MAEs) of the bone density prediction regression model in the test set and external testing dataset are 8.20 mg/cm3 and 9.23 mg/cm3, respectively, and the root mean square errors (RMSEs) are 10.25 mg/cm3 and 11.91 mg/cm3, respectively. The R-squared values are 0.942 and 0.923, respectively. The Pearson correlation coefficients are 0.972 and 0.965. The multi-feature fusion DCNN model based on only the lumbar 1 vertebrae and clinical variables can perform bone density three-classification diagnosis and estimate volumetric bone mineral density. If confirmed in independent populations, this automated opportunistic chest CT evaluation can help clinical screening of large-sample populations to identify subjects at high risk of osteoporotic fracture.

Joint learning for semantic segmentation and disparity estimation is adopted to scene parsing for mutual benefit. However, existing joint learning approaches unify the two task briefly which may result in negative feature mixing. In order to solve the problem, a win–win approach Stereo Semantic Network (SSNet) is proposed for pixel-wise scene parsing. SSNet is the first Transformer based end-to-end joint learning model for semantic segmentation and disparity estimation. The main novelty lies in the proposed Transformer Feature Separation Module (TFSM) which is designed to separate features for segmentation prediction and disparity regression according to the characteristics of the two tasks. The segmentation and disparity results are supervised jointly with a weighted summation loss function to improve the performance of both tasks. Experimental results on Cityscapes Dataset and KITTI 2015 Dataset demonstrate that SSNet outperforms state-of-the-art joint learning approaches.

To develop and evaluate a deep learning model based on chest CT that achieves favorable performance on opportunistic osteoporosis screening using the lumbar 1 + lumbar 2 vertebral bodies fusion feature images, and explore the feasibility and effectiveness of the model based on the lumbar 1 vertebral body alone. The chest CT images of 1048 health check subjects from January 2021 to June were retrospectively collected as the internal dataset (the segmentation model: 548 for training, 100 for tuning and 400 for test. The classification model: 530 for training, 100 for validation and 418 for test set). The subjects were divided into three categories according to the quantitative CT measurements, namely, normal, osteopenia and osteoporosis. First, a deep learning-based segmentation model was constructed, and the dice similarity coefficient(DSC) was used to compare the consistency between the model and manual labelling. Then, two classification models were established, namely, (i) model 1 (fusion feature construction of lumbar vertebral bodies 1 and 2) and (ii) model 2 (feature construction of lumbar 1 alone). Receiver operating characteristic curves were used to evaluate the diagnostic efficacy of the models, and the Delong test was used to compare the areas under the curve. When the number of images in the training set was 300, the DSC value was 0.951 ± 0.030 in the test set. The results showed that the model 1 diagnosing normal, osteopenia and osteoporosis achieved an AUC of 0.990, 0.952 and 0.980; the model 2 diagnosing normal, osteopenia and osteoporosis achieved an AUC of 0.983, 0.940 and 0.978. The Delong test showed that there was no significant difference in area under the curve (AUC) values between the osteopenia group and osteoporosis group (P = 0.210, 0.546), while the AUC value of normal model 2 was higher than that of model 1 (0.990 vs. 0.983, P = 0.033). This study proposed a chest CT deep learning model that achieves favorable performance on opportunistic osteoporosis screening using the lumbar 1 + lumbar 2 vertebral bodies fusion feature images. We further constructed the comparable model based on the lumbar 1 vertebra alone which can shorten the scan length, reduce the radiation dose received by patients, and reduce the training cost of technologists.

Despite the potential benefits of mobile health (mHealth) services and applications, older adults face challenges and barriers to adopt mHealth. Published work on opportunities and challenges of mHealth for aging China has focused on trends in mHealth and its clinical applications, but not older adults' perceptions or their processes of adoption. In this study, questionnaires and interviews were conducted to understand older adults' perceptions and adoption of mHealth services in China. Conceptual models were generated using existing theories and refined using the findings of the user research. The proposed "pine tree model" illustrates the factors influencing Chinese older adults' adoption of mHealth services and describes the process for better adoption. Based on the research, suggestions were proposed to promote mHealth adoption among older adults, which will inform better design of mHealth for the aging population in China.

Osteoporosis is a significant global health concern that can be difficult to detect early due to a lack of symptoms. At present, the examination of osteoporosis depends mainly on methods containing dual-energyX-ray, quantitative CT, etc., which are high costs in terms of equipment and human time. Therefore, a more efficient and economical method is urgently needed for diagnosing osteoporosis. With the development of deep learning, automatic diagnosis models for various diseases have been proposed. However, the establishment of these models generally requires images with only lesion areas, and annotating the lesion areas is time-consuming. To address this challenge, we propose a joint learning framework for osteoporosis diagnosis that combines localization, segmentation, and classification to enhance diagnostic accuracy. Our method includes a boundary heat map regression branch for thinning segmentation and a gated convolution module for adjusting context features in the classification module. We also integrate segmentation and classification features and propose a feature fusion module to adjust the weight of different levels of vertebrae. We trained our model on a self-built dataset and achieved an overall accuracy rate of 93.3% for the three label categories (normal, osteopenia, and osteoporosis) in the testing datasets. The area under the curve for the normal category is 0.973; for the osteopenia category, it is 0.965; and for the osteoporosis category, it is 0.985. Our method provides a promising alternative for the diagnosis of osteoporosis at present.

The way of constructing a robust feature pyramid is crucial for object detection. However, existing feature pyramid methods, which aggregate multi-level features by using element-wise sum or concatenation, are inefficient to construct a robust feature pyramid. The reason is that these methods cannot be effective in discriminating the relevant semantics of objects. In this article, we propose a Complementary Feature Pyramid Network (CFPN) to aggregate multi-level features selectively and efficiently by exploring complementary information between multi-level features. Specifically, a Spatial Complementary Module (SCM) and a Channel Complementary Module (CCM) are designed and embedded in CFPN to enhance useful information and suppress irrelevant information during feature fusions along spatial and channel dimensions, respectively. CFPN is a generic feature extractor, as evidenced by its seamless integration into single-stage, two-stage, and end-to-end object detectors. Experiments conducted on the COCO and Pascal VOC datasets demonstrate that integrating our CFPN into RetinaNet, Faster RCNN, Cascade RCNN, and Sparse RCNN obtains consistent performance improvements with negligible overheads. Code and models are available at: https://github.com/VIPLab-CQU/CFPN.

Neural dendrites play a critical role in various cognitive functions, including spatial navigation, sensory processing, adaptive learning, and perception. The spatial layout, signal processing, and nonlinear dynamics of dendrites endow them with these abilities. However, designing an efficient learning mechanism with spiking dendrites remains a challenging problem. In this article, a novel biologically plausible learning method is developed to address this challenge. The method uses a multi-scale learning rule with dendritic predictive characteristics. A two-phase learning mechanism based on burst-related plateau potential dynamics of spiking dendrites is utilized to achieve global learning of the spiking model. Experimental results demonstrate that the proposed algorithm can improve the learning accuracy and reduce the synaptic operations compared to the previous dendritic learning rule without dendritic predictive mechanism. It effectively reduces both the synaptic operations and the spike number in the output layer, leading to a reduction of power consumption on neuromorphic hardware. This suggests the multi-scale combination of the three-factor dendritic prediction principle and two-phase plateau potential activities can enhance the learning capability and sparsity within a single neuron. Besides, our learning method enhances the robustness and improves the learning convergence speed. We also explore different model variation formations of our learning model. The proposed study can contribute to spike-based machine learning and neuromorphic computing. It is also meaningful for a deeper understanding of the dendritic roles on biologically plausible credit assignment in the brain cortex.

Video object detection aims to detect and track each object in a given video. However, due to the problem of appearance deterioration in the video, it is still challenging to obtain good results when we apply traditional image object detection methods to videos. In this paper, we propose a new feature aggregation method, called Dual Feature Aggregation (DualFeat) for video object detection. By effectively combining the temporal and spatial attention mechanisms, we make full use of the temporal and spatial information in videos. Meanwhile, we leverage a real-time tracker to track detected objects in video frames, where features are aggregated again with previously obtained features. Such a way helps to obtain more comprehensive and richer features, greatly improving the accuracy of video object detection. We perform experiments on the ILSVRC2017 dataset, and the experimental results also verify the effectiveness of our method.

The performance of stereo vision tasks degrades when haze exists in the input stereo image pair. Independently applying single image dehazing algorithm on left and right images is not optimal. To overcome the problem, we propose an effective framework, called SRDNet, for simultaneously dehazing stereo images. The main idea of SRDNet is to make full use of the stereo information from cross views improving dehazing performance. It does not explicitly employ the disparity estimation and the correlation matrix. SRDNet comprises two parts: a weight-sharing coarse dehazing network (WSCDN) and a guided separated refinement network (GSRN). The WSCDN is utilized to predict a coarse dehazed image pair. Then the GSRN is introduced to predict the residues for different views by extracting the fused information of cross views and separating the features of different views with a guided channel and spatial refinement module. The residues are added to the coarse dehazed pair so as to make refinement and remove the remained haze. Experimental results demonstrate that our proposed SRDNet surpasses previous image dehazing methods by a significant margin both quantitatively and qualitatively. Moreover, our SRDNet could be a preprocessing step of the stereo-based 3D object detection and boosts the 3D detection accuracy in hazy scenes.

本文研究了无人机集群躲避动态障碍物下的队形控制问题.首先,引入针对动态障碍物的碰撞预判机制判断集群是否需要对障碍物进行规避.其次,在动态障碍物与无人机间构造斥力场实现避障.最后,根据一致性理论设计基于集群各无人机之间、无人机与虚拟领导者之间的位置、速度一致性控制律,结合人工势场法实现躲避动态障碍物下集群队形的形成与保持.仿真结果表明,集群无人机能够在以分布式方式躲避动态障碍物的同时实现队形的形成、保持与重构.

Most state-of-the-art deep learning based depth estimation methods follow the pipeline of firstly forming a 4D cost volume (feature dimension, max disparity, height, and width) and then regressing disparity from the cost volume by several 3D convolutional layers. Applying 3D operations on the 4D tensor leads to unacceptable computational complexity and memory cost. To solve the problem, we aim at replacing the 4D cost volume with 3D cost volume so that the disparity can be regressed by 2D convolutions to achieve a good balance between efficiency and effectiveness. To this end, a light-weighted network, called PCNet, is proposed to generate 3D cost volume. The main novelty lies in the proposed Paired Channel Feature Volume (PCFV) which is capable of combining the features of stereo pairs with specially designed 3D filters to preliminarily encode the relationship between each pair of the channels. Moreover, a densely connected aggregation on the outputs of PCFV is performed to exploit much richer contextual information. Experimental results on the SceneFlow, KITTI 2012, and KITTI 2015 datasets demonstrate that the proposed PCNet achieves comparable accuracy with state-of-the-art methods and keeps high efficiency as well.

Multi-task learning (MTL) paradigm focuses on jointly learning two or more tasks, aiming for significant improvement w.r.t model's generalizability, performance, and training/inference memory footprint. The aforementioned benefits become ever so indispensable in the case of joint training for vision-related {\bf dense} prediction tasks. In this work, we tackle the MTL problem of two dense tasks, i.e., semantic segmentation and depth estimation, and present a novel attention module called Cross-Channel Attention Module ({CCAM}), which facilitates effective feature sharing along each channel between the two tasks, leading to mutual performance gain with a negligible increase in trainable parameters. In a true symbiotic spirit, we then formulate a novel data augmentation for the semantic segmentation task using predicted depth called {AffineMix}, and a simple depth augmentation using predicted semantics called {ColorAug}. Finally, we validate the performance gain of the proposed method on the Cityscapes and ScanNet dataset, which helps us achieve state-of-the-art results for a semi-supervised joint model based on depth and semantic segmentation.

Semantic segmentation is a challenging task in computer vision which is widely used in autonomous driving and scene understanding. State-of-the-art semantic segmentation networks, like DeepLab and PSPNet, make full use of multiple feature information to improve spatial resolution. However, the feature resolution in the scale-axis is not dense enough for practical applications. To tackle this problem, a multi-stream network is designed with atrous convolutional layers at multiple rates to capture objects and context at multiple scales. Furthermore, intra-connections and inter-connections are designed to fuse multi-scale features densely which produce a feature pyramid with much larger scale diversity and larger receptive field by involving small quantity of computation. The proposed module can be easily used in other methods and it helps to increase the performance. Compared with existing methods, the proposed network, called Multi-stream Densely Connected Network, reaches competitive results on ADE20K dataset, PASCAL VOC 2012 dataset, and Cityscapes dataset.

Electroencephalogram (EEG) has been widely used for the detection of anxiety because of its ability to reflect the functional activities of the brain. However, EEG alone may not provide precision in the detection of anxiety because other emotional disorders usually trigger the same changes in brain function. To discover effective diagnostic indicators and to achieve more precise anxiety detection, we integrate eye movement information into EEG and divide the features into groups according to their respective characteristics. Then, we use group sparse canonical correlation analysis (GSCCA) to investigate group structure information among EEG and eye movement features and obtain an effective fusion representation of EEG and eye movement to achieve more precise detection of anxiety mood. The experimental results from 45 anxious subjects and 47 normal controls from the Healthy Brain Network (HBN) dataset showed that GSCCA could be effectively used to explore the correlation between EEG features within different scalp regions and eye movement features from several aspects. Visual behaviors, including saccades and fixation, are more linearly related to the power spectrum of EEG on the scalp area corresponding to the visual region of the brain. The ultimate fusion representation achieved an optimal classification accuracy of 82.70 percent with the support vector machine (SVM) classifier on the gamma band of EEG.

Self-supervised monocular depth estimation has seen significant progress in recent years, especially in outdoor environments. However, depth prediction results are not satisfying in indoor scenes where most of the existing data are captured with hand-held devices. As compared to outdoor environments, estimating depth of monocular videos for indoor environments, using self-supervised methods, results in two additional challenges: (i) the depth range of indoor video sequences varies a lot across different frames, making it difficult for the depth network to induce consistent depth cues for training; (ii) the indoor sequences recorded with handheld devices often contain much more rotational motions, which cause difficulties for the pose network to predict accurate relative camera poses. In this work, we propose a novel framework-MonoIndoor++ by giving special considerations to those challenges and consolidating a set of good practices for improving the performance of self-supervised monocular depth estimation for indoor environments. First, a depth factorization module with transformer-based scale regression network is proposed to estimate a global depth scale factor explicitly, and the predicted scale factor can indicate the maximum depth values. Second, rather than using a single-stage pose estimation strategy as in previous methods, we propose to utilize a residual pose estimation module to estimate relative camera poses across consecutive frames iteratively. Third, to incorporate extensive coordinates guidance for our residual pose estimation module, we propose to perform coordinate convolutional encoding directly over the inputs to pose networks. The proposed method is validated on a variety of benchmark indoor datasets, i.e., EuRoC MAV, NYUv2, ScanNet and 7-Scenes, demonstrating the state-of-the-art performance.

BACKGROUND:Advances in mobile technology and public needs have resulted in the emergence of mobile health (mHealth) services. Despite the potential benefits of mHealth apps, older adults face challenges and barriers in adopting them.OBJECTIVE:The aims of this study are to understand older adults' perception of mHealth services and to discover the barriers that older adults face in the initial adoption of mHealth apps.METHODS:This paper systematically analyzed main determinants related to mHealth services and investigated them through questionnaires, interviews, and a workshop. Two studies were carried out in London. In study 1, the questionnaires with follow-up interviews were conducted based on the literature review to uncover older adults' perception (including perceived usefulness, perceived ease of use, and perceived behavioral control) of mHealth services. Study 2 was a workshop helping older adults to trial selected mHealth apps. The workshop was conducted by the first author (JP) with assistance from 5 research students. The barriers that older adults faced in the initial adoption period were observed. The interviews and workshop were audiotaped and transcribed. Descriptive statistics and the thematic analysis technique were used for data analysis.RESULTS:In total, 30 older adults in London completed the questionnaires and interviews in study 1. The results of study 1 show that the lack of obvious advantage, low reliability, scary information, and the risk of privacy leakage would decrease older adults' perceived usefulness of mHealth services; the design of app interface would directly affect the perceived ease of use; and aging factors, especially the generation gap, would create barriers for older users. In total, 12 participants took part in the workshop of study 2, including 8 who took part in study 1. The results of study 2 identified that access to technology, the way of interaction, the risk of money loss, heavy workload of using an mHealth app, and different lifestyle are influential factors to older adults' adoption of mHealth services.CONCLUSIONS:The perceptions of mHealth services of older adults were investigated; the barriers that older adults may face in the initial adoption stage were identified. On the basis of the synthesis of these results, design suggestions were proposed, including technical improvement, free trial, information clarification, and participatory design. They will help inform the design of mHealth services to benefit older adults.

Driver fatigue has become a leading cause of accidents and death in the transportation industry. Electroencephalography (EEG)-based fatigue detection can be a good way to reduce accidents and improve safety and efficiencies throughout the transportation system. In this article, we focus on investigating whether the spatial–temporal changes in the relations between EEG channels are specific to different driving states. EEG signals were first partitioned into several segments, and the covariance matrices obtained from each segment were input into a recurrent neural network to extract high-level temporal features. Meanwhile, the covariance matrices of whole signals were leveraged to extract spatial characteristics that were fused with temporal features to obtain comprehensive spatial–temporal information. In experiments on an open benchmark dataset, our method achieved an excellent classification accuracy of 89.28% and showed superior performance compared to several other state-of-the-art methods. These results indicate that our method can enable higher performance in driver fatigue detection.