This paper presents a self-supervised feature learning method for hyperspectral image classification. Our method tries to construct two different views of the raw hyperspectral image through a cross-representation learning method. And then to learn semantically consistent representation over the created views by contrastive learning method. Specifically, four cross-channel-prediction based augmentation methods are naturally designed to utilize the high dimension characteristic of hyperspectral data for the view construction. And the better representative features are learned by maximizing mutual information and minimizing conditional entropy across different views from our contrastive network. This 'Cross-View-Predicton' style is straightforward and gets the state-of-the-art performance of unsupervised classification with a simple SVM classifier.
Flow classification is a key technique in network encryption machine, which matches 5-tuples of IP packet against a predefined rule-set to decide the subsequent processing, such as secret transmission\clear transmission\discard. Currently, there is no flow classification algorithm that can support both 10Gbps line-rate range matching and rule set real-time modification on the CPU. We solve this problem through FPGA and CPU cooperation. We perform range matching on the first packet of the data flow on the FPGA, and create a flow table entry on the CPU side. Then perform the flow table hash matching on the subsequent packets on the CPU, and design the synchronous updating scheme of the rule set and the flow table. The mechanism can significantly reduce the computational and communication overhead.
Sign language is a bridge for communication be-tween people with normal hearing and impaired hearing. How-ever, only a few people with normal hearing have an understanding of sign language. Thus, Sign Language Recognition (SLR) has attracted significant interest from both industry and academia. Prior traditional methods have certain limitations such as light condition and extra hardware. To address these issues, this study aims to recognize sign languages using acoustic signals on a smartphone. We design an SLR system called SignGest, which captures user's sign language gestures with built-in microphones. Afterwards, we build a Convolutional Neural Network(CNN) model to extract features of different gestures to distinguish them. Furthermore, we use a Deep Convolutional Generative Adversarial Network(DCGAN) to generate abundant training data that look no different from the real samples. We handle various challenges including effective gesture segmentation and training data collection. Finally, SignGest is implemented on a server and an Android smartphone with built-in microphones and speakers without any extra hardware or infrastructure equipment. When users perform the sign language gestures before the smartphone, SignGest can recognize each of them and print the result. Through extensive experiments, our results show that SignGest can achieve robust and satisfactory performance.
The widespread application of specialized embedded devices and their ever-growing sensing capabilities lead to an overload of raw data reaching the Cloud via the Internet of Things. Despite this massive amount of available data, data-driven applications still produce a limited impact on the real world. We argue that this limitation arises from the complexity the developers still face when engineering and deploying solutions on a plurality of embedded devices. Instead of focusing on constructing valuable information from raw data and designing algorithms to exploit it, the developers still have to figure out on their own: how to partition the computation between Edge, Fog, and Cloud; how to move partial results around in highly distributed solutions; how to create abstract interfaces to expose information-rich endpoints. This paper introduces the Virtual Sensor Domain-Specific Language (DSL), which builds the keystone for designing and developing powerful Computer-Aided Design tools and Integrated Development Environments to support the developers in focusing on information manipulation. The Virtual Sensor DSL is built on top of C++ to ease its adoption by the embedded development community. It introduces a reactive approach to data collection, technology mapping, and workload distribution. It allows the creation of adaptable Virtual Sensors, streaming processed information coming from the integration of multiple raw data streams, and it comes with integrated simulation capabilities, supporting prototyping. In this paper, we describe the entities the language offers, how the developers can exploit them, and analyze a real-world use case that benefitted from the Virtual Sensor DSL.
In this paper, we consider an integrated terrestrial-aerial access network where multiple access points (APs) such as cellular base stations (BSs), unmanned aerial vehicles (UAVs), and high altitude platforms (HAPs) coordinate to serve various ground users. Advanced forms of typical data services for 5G and beyond such as enhanced mobile broadband (eMBB), massive machine-type communications (mMTC), and ultra-reliable low-latency communications (URLLC), are provided via network slicing (NS) to cope with the heterogeneity brought by the terrestrial-aerial integration. To satisfy various requirements of users, network slices consist of different virtual network functions (VNFs), each of which consumes different amounts of compute resource at APs. To achieve a tradeoff between data rate and consumed compute resource of VNFs, we aim to maximize the compute resource utilization efficiency by jointly optimizing the VNF deployment, power and spectrum resource allocation. The mutual relation between VNF deployment and user scheduling is modelled, based on which an iterative algorithm is proposed to solve the problem. Simulation results show that the proposed algorithm achieves a better tradeoff than benchmark ones. Influence of the VNF deployment and size on the system performance is revealed as well.
Safety-critical real-time embedded systems are becoming difficult to model and analyze as they accommodate more and more functionalities, and they rely on more and more complex HW and SW implementations. One of the main difficulty with such systems is to find the right coupling between the task model and the analysis method. Indeed, if the task model is too simple it may add pessimism to the result of the analysis, and it may not be possible to prove the correctness of the systems. If the model is too complex, the analysis may be not efficient enough to be usable; or worst the model may have an incomplete method only or no method at all, so the system cannot be analyzed. Finding, the right couple task model analysis method is a balance between expressiveness and analysis efficiency. In this article, we introduce a new graph real-time task model (noted GGTM) which has been used for several years for the deployment and the analysis of different embedded systems. We discuss its expressiveness compared to the literature and demonstrate its advantages. We also, present a decidable and efficient, exact schedulability analysis.
A smart contract is a program running on a blockchain platform. Smart devices send data to smart contracts or change their own status based on smart contracts. Businesses want to use smart devices and smart contracts to streamline workflow because smart contracts reduce the need in trusted intermediators and cut down enforcement costs. However, developing smart contract applications is challenging due to different memory models, different interaction models, and a dearth of supporting tools and libraries. When developers are asked to reimplement conventional applications to smart contracts, it is thus ideal to automatically transform them, avoiding manual labor, as well as ensuring reliability and security. This paper contributes a set of rules to transform conventional applications in the Model-View-Controller (MVC) pattern into smart contracts running on Hyperledger Fabric, a blockchain platform preferred by businesses. Major transformations are performed in the model, while in the controller, model calls are replaced by smart contract calls. The source application and the target smart contract are all written in Java. Our rules add read-your-writes consistency that Hyperledger Fabric does not natively support. Runtime pre- and post-condition checking in the original application is supported in the transformed smart contract. We evaluated our rules on CoCoME and other MVC applications, and all code ran correctly and passed unit tests.
Multiprocessor real-time operating systems (RTOSs) are in high demand to deal with the large complexity of embedded systems. Developers can reduce power consumption, cope with increased size, and improve processing performance using multiprocessor RTOSs. However, multiprocessor RTOSs for embedded systems have two major problems. The handling of kernel objects, functionalities of Multiprocessor RTOSs, makes system reusability difficult. The same goes for the low reusability of interprocessor communication in multiprocessor systems. To solve these problems, this paper proposes a component framework for multiprocessor RTOSs. Kernel objects were componentized based on TOPPERS embedded component systems, a component-based development framework. The proposed framework supports object allocation on multiprocessors, and implements automatic code generation plugins for system reusability and flexible interprocessor communication methods depending on the system. The reusability of the system and inter-processor communication will be verified to be easy while maintaining real-time performance and resource constraints.
Vehicular Edge Computing (VEC) is a computing paradigm that brings Mobile Edge Computing (MEC) to the road and vehicular scenarios by providing low-latency and high-efficiency computation services. One key technology of VEC is task offloading, which allows vehicles to send computation tasks to surrounding Roadside Units (RSUs) for execution, thereby reducing service delay. However, the existing task offloading schemes face the important challenges because the vehicles with time-varying trajectories and limited computing resources need to process massive data with high complexity and diversity. In this paper, we propose a Cooperative Task Qffloading Scheme (CTOS) based on Cybertwin-assisted VEC. Specially, a novel Cybertwin-assisted VEC network architecture is established by applying the combination of the Digital-Twins (DT) and the Generative Adversarial Network (GAN). With the powerful prediction capability of GAN, the data of DT is advanced with the physical entity, which is an effective assistant for task offloading. Then, we leverage the distributed Deep Reinforcement Learning (DRL) to make offloading decisions, which consider the limited resources of RSUs and the cooperation of vehicles. The simulation results demonstrate that the proposed scheme can achieve excellent performance in terms of system stability and efficiency.
An intelligent military blockchain security management architecture is proposed. Relying on the hierarchical architecture of the data layer, network layer, consensus layer, contract layer and application layer of the blockchain system, the multidimensional detection, intelligent analysis and evaluation, and visual security early warning of abnormal behavior data are carried out by using big data and artificial intelligence technology with real-time collection of various original data information such as nodes, chains, contracts, transactions, accounts and consensus, and security events are quickly located and handled, so as to realize the intelligent linkage closed-loop process of military blockchain security management.
For practical industrial wireless sensor networks (IWSNs), the system freshness of a specific task is usually related to multiple and multitype sensing data. However, most existing research on freshness metrics, such as Age of Information (AoI) or Age of Processing (AoP), only considers a single-package setting with a single type of data. To fill this gap, we propose the Age of Task-oriented Information (AoTI) for measuring the freshness of industrial tasks in IWSNs. It measures the time elapsed of the latest analyzed results before arriving at the receiver since the generation of any type of sampling data belonging to one certain task. Furthermore, we aim to minimize the long-term AoTI for IWSNs applications by jointly optimizing access modes and sampling frequencies for all sensors. By first formulating the problem as a Mixed Integer Nonlinear Program-ming problem, we then transform it to a constrained Markov Decision Process (CMDP) and relax it as an un-constrained MDP using Lagrangian method. Finally, we develop a Learning-based Access mode selection and Sampling frequency Control (LASC) algorithm and verify its superiority through simulations.
Vision-based maritime surveillance has become an essential part of the vessel traffic services system. The images collected in low-light maritime conditions often suffer from poor visibility. These images may significantly degenerate the performance of high-level visual tasks and increase the uncertainty in maritime surveillance. To address this problem, we propose a lightweight neural network (Rep-Enhancer) for low-light image enhancement. Specifically, we first design a re-parameterizable multi-branch edge extraction module, i.e., spatial domain-oriented convolution block (SDCB). Furthermore, skip connections and spatial attention operations are employed to strengthen the features. By exploiting these well-strengthened edge features, we can enhance the low-light images effectively with the encoder-decoder structure. The experimental results have shown that Rep-Enhancer can enhance the low-light image qualifiedly while maintaining great inference efficiency.
It is increasingly popular to utilize the wisdom of the crowd for knowledge discovery and monetization. Most of the existing knowledge marketplaces in crowdsensing are implemented by a third-party platform, which may compromise users' rights and be vulnerable to incurring attacks in practice. To eliminate the untrustworthy behaviors of the third party and improve tolerance for the attacks, some blockchain-based knowledge marketplaces in crowdsensing have been proposed. However, the existing blockchain-based knowledge marketplaces fail to simultaneously guarantee privacy (i.e., data privacy and task privacy) and quality awareness. In this paper, we design a blockchain-based privacy-preserving quality-aware knowledge marketplace (PQKM) based on truth discovery, secure K-nearest neighbor computation, matrix decomposition, and data perturbation. PQKM privately calculates users' data quality and automatically rewards users based on their data quality. Detailed security analysis demonstrates that PQKM can preserve data privacy and task privacy during knowledge discovery and monetization. Extensive experiments are conducted on the open real-world dataset to show that PQKM has acceptable efficiency and affordable performance.
The mission-critical decisions are usually made by a central node in connected autonomous systems enabled by IoT, AI and 5G etc. The reliability of decision-making largely depends on the condition of the central node, which can have unaffordable costs and low scalability in a massive-scale mobility network. Therefore, mission-critical IoT networks have been seeking new methods to achieve the growing reliability and latency requirement of cooperative decision-making. The distributed consensus protocol, which has been widely used in distributed computing systems, can provide robustness to the liveness of mission-critical decisions that may encounter node faults. This article first analytically derives the performance of two distributed consensus protocols: Raft and Hotstuff BFT, with the extra synchronization phases. In the presence of communication failure, the results are compared with the analytically derived performance of a centralized consensus. The comparison indicates the strengths and weaknesses of these consensus mechanisms from the perspective of full consensus reliability and communication latency, which provides guidelines for deploying the appropriate distributed autonomous systems in the future.
It is important to obtain accurate indoor location information in the Wi-Fi fingerprinting localization system, which suffers from a lot of labour-power to construct the fingerprint database. Improving positioning accuracy with less manual labour is still an issue. In this paper, we propose a signal-physical Siamese neural network SPSNN, consisting of a signal distance matrix encoder and a physical distance matrix encoder to learn features. The main idea of the SPSNN is to use the physical feature to constrain the signal feature. Grid fingerprints are constructed using crowdsourcing samples. We compute the distance difference between the sample and different grid fingerprints in signal space to build the pictures as the input of the Siamese network for further feature extraction. In addition, we exploit the distance distribution information in physical space to modify and constrain the signal feature vector during the network training phase. The position prediction is given by the regression method. Experiments on-field measurements validate the effectiveness and robustness of the proposed model, and the optimal localization accuracy is less than 1 m.
Smart Parking has gained so much popularity in recent years due to the increasing number of vehicles in big cities, resulting in traffic congestion in urban areas. Not only on the streets but also in places such as educational institutions, hospitals, commercial activities, special events, and entertainment uses. Finding a free parking lot in these places has evolved difficulty for the drivers. To solve such a problem, governments and researchers tried to find alternative solutions to overcome or mitigate the traffic congestion. Many solutions have been proposed, such as increasing the parking capacities, which takes much time or makes it hard to find a square area in crowded places. Most existing studies, do not consider the cost of deployment, energy, and time-tomarket consideration which makes the available systems need further investigation. In this paper, we propose an intelligent parking system prototype that can be useful for the drivers to have a prior knowledge about the available parking lots in the area of interest. Our proposed system involves deploying mmWave Radar sensor nodes in each parking lot to detect the availability of parking vehicle through transmitting radio pulses periodically. The detected information can be sent to the gateway through multi-hop for further statistics and reports. We also give an intensive analysis and study about the challenges and consideration on the mmWave radar design aiming to improve the detection accuracy and avoid falsedetection that occurs from objects near to the sensor. To ensure continuous operation and extend sensor life-time, we propose Bluetooth Low Energy BLE-enabled relay-feature as the communication protocol between the nodes.
With the advent of Industry 4.0, Cyber Physical Systems (CPS) and Internet of Things (IoT) technology provide enormous opportunities and support for intelligent manufacturing. Material handling is vital in manufacturing systems to ensure that proper materials with the right quantity and quality can be delivered to each machine or workstation at the right time. AGV has been widely used in smart factories for material handling, and AGV scheduling plays a critical role in practical AGV application. However, the AGV scheduling problem becomes more and more complex with the development of intelligent manufacturing, making the design of effective and efficient scheduling algorithms complicated. In this paper, we analyzed the AGV dispatching tasks in the workshop and model the workshop as a node network, and apply an improved Multi-Agent Reinforcement Learning (MARL), that is, an improved QMIX model to solve the AGV scheduling problem. The experiment results show that the proposed approach outperforms the other commonly-used methods such as deep reinforcement learning (DQN) under different environment settings, in term of the maximum makespan of AGV.
Port traffic flow modeling based on big data is an important research direction in the shipping field, having the task of traffic forecasting for ports worldwide. Graph neural networks have a strong ability to capture the spatial topology characteristics and may be combined with recurrent neural networks or dilated convolution methods in time series prediction, producing a large number of spatiotemporal graph convolution models. Such models have been widely and successfully applied in traffic forecasting. Differing from urban traffic flow data, the statistical time span of port vessel flow and throughput data is large, its spatial span is wide, and the data experience significant fluctuations. Consequently, certain spatiotemporal graph convolution traffic prediction models are unsuitable for shipping scenarios. To address this shortcoming, we have created a unique port flow dataset based on automatic identification system (AIS) and port geographic data. Using theoretical analysis and experimental comparison, we have determined the most appropriate model for shipping predictions based on existing spatiotemporal graph models and have proposed model optimization recommendations for the maritime domain. Our experiment based on an open source traffic forecasting framework to compare the results of multiple existing spatiotemporal graph models under fair conditions with the central ports of Rotterdam, Shanghai, Boston, and Singapore. The results show that Graph WaveNet exhibits better performance in shipping scenarios.
This paper presents a tool to evaluate the latency of Robot Operating System (ROS) 2 applications called Chain-Aware ROS 2 Evaluation Tool (CARET). ROS 2 is designed to enhance the modularity of real-time robotic applications, including self-driving software such as Autoware. To analyze the performance of ROS 2 applications, CARET supports measurement functionalities for the callback latency, node latency, communication time between nodes, and end-to-end latency. To calculate each latency, CARET provides tracepoints, an architecture file, information on tracepoint connections, and a message tracking functionality. Furthermore, CARET can visualize different types of latency, such as bottlenecks and lost message, to analyze ROS 2 applications. The experimental results demonstrate that CARET can successfully measure the end-to-end latency of Autoware.Universe, which is ROS 2-based self-driving software.
After the COVID-induced lock-downs, augmented/virtual reality turned from leisure to desired reality. Real-time 3D audio is a crucial enabler for these technologies. Nevertheless, systems offering object spatialization in 3D audio fall in two limited cases. They either require long-running pre-renders or involve powerful computing platforms. Furthermore, they mainly focus on active audio sources, while humans rely on the sound's interactions with passive obstructions to sense their environment. We propose a hardware co-processor for real-time 3D audio spatialization supporting passive obstructions. Our solution attains similar latency w.r.t. workstations while draining a tenth of the power, making it suitable for embedded applications.
