With the expansion of applications and areas of blockchains, there is an increasing need for faster transaction validation platforms. The Practical Byzantine Fault Tolerance (PBFT) algorithm is one of the best validation processes to meet the need for velocity and security of successful consensus. Despite the implementations in several platforms such as Hyperledger Fabric, the performance does not reach what was hoped for their consensuses. The objective of this article is to present a performance evaluation model based on an Open Queuing Network to study the limits of PBFT and pipelined PBFT and to succeed in presenting a new parallelization approach in the form of parallel clusters. The advantage of this model is that it closely evaluates the different stages of PBFT execution according to the load submitted by the clients. This model is validated with the ns-3 simulator and has the advantage of quickly reaching evaluations that the simulator is limited in time to have them. The results show the importance of having consensus executions on parallel clusters to distribute the loads between nodes and clients. With cluster-based parallel PBFT, we can reach even twenty thousand transactions per second close to the validation values of Visa transactions. This opens the horizons towards the effective use of blockchains for new types of fast applications.
Blockchain is a distributed ledger that various applications can use to store data in a secure, decentralized, and immutable manner. The key piece in a blockchain is the consensus algorithm that different blockchain nodes run to obtain mutual agreement. While the Practical Byzantine Fault Tolerant (PBFT) is one of the most used algorithms and provides several advantages, it has a high communication cost, and it is executed in a sequential manner for consistency. To improve PBFT for blockchain consensus use, related works have added restrictive assumptions that can compromise security, consistency, and fault tolerance capabilities. This paper proposes a novel parallel PBFT-based consensus algorithm called Random-Cluster parallel PBFT. Our algorithm runs parallel consensus on random clusters in the blockchain network and then the resulting consensus is broadcasted to the blockchain network for a global consensus. Our proposal reduces the communication costs and improves the performance of the consensus which makes the blockchain more scalable. We provide formal proofs of the correctness of our algorithm. We have also proposed an algorithm to resolve the transaction order problem to address a consistent distributed ledger with concurrent consensus. The evaluation of throughput and communication complexity shows that Random-Cluster parallel PBFT provides better performance and more relevant results.
Recently, distributed deep learning has been introduced as the new highly computational solution that could handle huge amounts of data and reduce training time. Especially when handling high-dimensional and complicated data, is very challenging, such as dealing with Genomics which is the most demanding in terms of data acquisition, storage, distribution, and analysis. However, Distributed deep learning has issues that need to be resolved. Focusing on the synchronization paradigm, BSP (Bulk Synchronous Parallel) is the most used model. Even so, it is demanding in terms of time due to an exigent problem called the straggler, where all the workers need to wait for the slowest worker to synchronize. Therefore, in this article, we propose a collaborative BSP (Collab-BSP) that aims to solve this issue by adopting LSTM for execution time prediction and implementing it with the Apache Spark environment. We proved the efficiency of our approach in reducing the waiting time and iteration time by 50% and 30%, respectively. Also, our approach demonstrated promising results while training a distributed CNN for protein classification with 98.82% accuracy and proved its capability to enhance distributed deep learning training.
Nowadays, all sectors utilize devices that are part of the Internet of Things (IoT) for the purpose of connecting and exchanging information with other devices and systems over the Internet. This increases the diversity of devices and their working environments, which, in turn, creates new challenges, such as real-time interaction, security, interoperability, performance, and robustness of IoT systems. To address these, many applications protocols were adopted and developed for devices with constrained resources. This paper surveys communication protocols divided according to their goals along with their merits, demerits, and suitability towards IoT applications. We summarize the challenges of communication protocols as well as some relevant solutions.
With the success of blockchains for providing a high security platform for distributed systems, new challenges are presented in relation with the deployment of this technology for IoT applications with time constraints. In this paper, we present a new lightweight blockchain platform that is scalable to hold a large number of transactions and with acceptable latency times. We have designed a new rapid consensus algorithm that improves PBFT to apply it in a random cluster with a small number of Miners. This will enable multiple consensus on different clusters to be executed in parallel to reduce the complexity of PBFT, have acceptable latency and increase the scalability. The evaluations carried out on our proposal by comparing it with other pipeline PBFT improvement approaches, show that our approach has better latency that is still accessible even with a high load of transactions.
Cancer, a centuries-old challenge in medicine, is now understood as a collection of diseases driven by unique genetic mutations [1] [2]. Identifying these variants is pivotal. Somatic variant calling is a process used in genomics to detect DNA mutations by comparing the genetic data of normal (healthy) cells with that of potentially mutated cells, typically from a tumor, and it is essential for understanding cancer development, diagnosis, and tailoring therapies. Accurate identification is a critical step [3], especially after Next-generation sequencing technologies emerged, enabling the production of genome sequences at an unprecedented rate [4]. Variant calling pipelines streamline the process; nevertheless, it is not without challenges. One of the primary issues is handling the vast genomic data efficiently. Traditional large-scale computation is typically processor-bound, which is suitable for conventional applications. However, in the big data world, where vast amounts of data are generated, genomics in our case, disk latency becomes a significant bottleneck. To address this, a new framework based on distributed computing is necessary. This framework is designed to scale from single servers to multiple machines, collectively known as a cluster. The core idea is to distribute the data across a large cluster of machines and bring the program to the data. This approach ensures that the entire cluster is utilized for both reading and processing the data [5]. This ensures faster somatic mutation identification, which is vital as these variants can provide valuable insights into the exact type of cancer, its aggressiveness, and in turn, potential treatment options. Therefore, speeding up the detection process can lead to early diagnosis and earlier interventions that could substantially improve survival rates. Additionally, compared with germline variations, the detection of somatic variants in the cancer genome is complex due to tumor purity (proportion of tumor cells relative to normal cells in the tumor) and tumor heterogeneity (different tumor cells can have different mutations) [6]. So enhancing the accuracy of these calls minimizes the chances of false positives (incorrectly identifying a mutation) and false negatives (missing a mutation), ensuring that patients receive the correct treatment
The Internet of Things (IoT) establishes connectivity between billions of heterogeneous devices that provide a variety of essential everyday services. The IoT faces several challenges, including energy efficiency and scalability, that require consideration of enabling technologies such as network softwarization. This technology is an appropriate solution for IoT, leveraging Software Defined Networking (SDN) and Network Function Virtualization (NFV) as two main techniques, especially when combined with Machine Learning (ML). Although many efforts have been made to optimize routing in softwarized IoT, the existing solutions do not take advantage of distributed intelligence. In this paper, we propose to optimize routing in softwarized IoT networks using Federated Deep Reinforcement Learning (FDRL), where distributed network softwarization and intelligence (i.e., FDRL) join forces to improve routing in constrained IoT networks. Our proposal introduces the combination of two novelties (i.e., distributed controller design and intelligent routing) to meet the IoT requirements (mainly performance and energy efficiency). The simulation results confirm the effectiveness of our proposal compared to the conventional counterparts.
Message queuing telemetry transport (MQTT) is an application layer protocol that enables effective device communication in the Internet of Things (IoT). MQTT operates according to a publish-subscribe model, where a broker receives messages from the publishers and then forwards them to the subscribers. However, existing cloud-based MQTT brokers lead to network bottlenecks due to the large number of devices that interact with them. Therefore, mist computing is involved, where the MQTT brokers are deployed closer to the IoT devices and the workload is distributed across multiple brokers. Nevertheless, the distributed mist-based MQTT architecture introduces serious issues caused by the massive volume of flow exchanged by IoT devices. In this article, we introduce MQTT-SD to address these issues. MQTT-SD enhances MQTT protocol syntax with fusion semantic aggregation. It accomplishes MQTT topic fusion on the MQTT broker side with consideration of topics' distribution. We carried out extensive simulation and emulation to demonstrate MQTT-SD efficacy. The results show that MQTT-SD reduces MQTT flow and has the lowest traffic load compared to MQTT+ and MQTT-MFA.
In the last decade, Distributed deep learning has been widely used and introduced in research for highly computational tasks where time is very critical due to its capability to train deep learning models in less time within High-performance computing (HPC) clusters. However, Distributed Deep learning faces issues in heterogeneous HPC environments such as with synchronization that could slow down the training and affect the quality of training, especially in real-time applications. Therefore, in this article, we aim to improve the quality of training the distributed deep learning by using the multivariate LSTM model to predict the execution time of workers relying on its capability of capturing long-term dependencies in sequence prediction problems. Our LSTM model showed good results during training, validation, and testing with RMSE = 0.5284, with a dataset of a trace of training jobs running ML algorithms in Alibaba PAI. Also, we compared our model with CNN1D and GRU well-known models for regression problems and our model showed better results compared to them.
The MQTT (Message Queuing Telemetry Transport) protocol is widely recommended for IoT (Internet of Things) communication. It uses a single broker for data exchange, which is typically placed in the cloud. However, this centralized approach is not effective for scaling, leading to mist and edge broker distribution. In distributed infrastructures, administrators are forced to manually reconfigure all MQTT communication to meet performance metrics. However, manual reconfiguration has several drawbacks, e.g., it is a time-consuming process and prone to error. To remedy these issues, this paper proposes combining Software-Defined Networking (SDN) and the MQTT protocol. This proposal enables dynamic reconfiguration of MQTT communication on distributed MQTT-based infrastructure. In the current paper, a realcase study of experiments on a real test-bed is carried out to justify the necessity of the proposed approach.
Somatic variant calling is the process of identifying mutations unique to individual cells that are not inherited through the germline. This technique is important for characterizing cancer samples and is naturally challenging, as somatic mutations often occur at very low frequencies. Yet its identification with precision and accuracy is essential for understanding cancer development and tailoring therapies. Furthermore, given the large-scale nature of genomic data, somatic variant calling can be a time-consuming process. Therefore, it is crucial to execute this process in a way that ensures low latency. To address these problems, we deployed the Halvade Somatic Variant Calling pipeline on a distributed cluster based on Apache Spark, to acquire the somatic mutations. Then, we optimized the detected mutations through a 1DCNN_filter, a Convolutional Neural Network (CNN) based filter designed to filter out false positives. Experiments demonstrated that our 1DCNN_Filter can filter out 93.4% of false positive Single Nucleotide Variants (SNVs) while still retaining 87.6% of true positives. Our solution allows for more True Positives to be retained compared to the existing Multi-Layer Perceptron (MLP) based filter.
The Internet of Things (IoT) is an emerging technology that aims to connect heterogeneous and constrained objects to each other and to the Internet. It has grown significantly in a wide variety of applications such as smart homes, smart cities, smart vehicles, etc. The huge number of connected devices increases the challenges, as IoT provides diverse and complex network services with different requirements on a common infrastructure. Network Softwarization is the latest network paradigm that transforms traditional network processes to the separation of hardware and software by using some enabling network technologies such as Software Defined Networking (SDN) and Network Function Virtualization (NFV). Machine Learning (ML) plays an essential role in creating smarter IoT networks, as it has shown remarkable results in various domains. Given that the network softwarization allows it to be easily integrated, ML can play a crucial role in efficient and self-adaptive IoT networks. In this paper, we provide a detailed overview of the concepts of IoT, network softwarization, and ML, and we study and discuss the state of the art of intelligent ML-enabled network softwarization for IoT. We also identify the most prominent future research directions to be considered.
Recently, blockchain technology has emerged as a revolutionary innovation with its distributed ledger feature that allows applications to store and transmit data in a secure, transparent, and immutable manner. One of the main ideas of blockchain technology is the consensus mechanism for reaching an agreement on the state of the distributed ledger. In this context, Practical Byzantine Fault Tolerance (PBFT) is one of the most popular algorithms. However, it has a high communication overhead and poor scalability. To overcome these limits, this paper presents the effectiveness of our algorithm Random Cluster PBFT (RC-PBFT) through an evaluation of its performance under various network conditions. Our algorithm runs on randomly selected clusters, and the results are then broadcast on the blockchain network. The effectiveness of RC-PBFT is demonstrated by implementations on the NS-3 network simulator, and the tests performed show that our approach achieves significant improvements compared to the original PBFT algorithm.
MQTT (Message Queuing Telemetry Transport) has become the perfect messaging protocol for IoT (Internet of Things) systems since it is the lightest protocol designed for low bandwidth, high-latency, unreliable networks. Today, the strategy of distributing several MQTT brokers on the networks is widely used because the strategy of using a single broker is no longer efficient. However, in the distributing architectures of MQTT brokers, a subscriber should have prior knowledge about the address of the broker that publishes the data on the topics of interest. In this paper, we tackle this challenge by proposing a mechanism that connects the subscribers to the brokers in a transparent way. The proposed approach, known as TD-MQTT (Transparent Distributed MQTT brokers), requires no prior knowledge of the brokers by the subscribers. The data will be carried automatically from brokers that can change their configuration and location. The transparency will help to use IoT data without worrying about their location and dynamic configuration changes. To evaluate our approach, we compared it with the basic distributed MQTT and the EMMA (MQTT Middle-ware for Edge Computing Applications) approach. The results of the evaluation show that TD-MQTT is much better than the standard MQTT, especially in terms of response time.
Big Data works perfectly along with Deep learning to extract knowledge from a huge amount of data. However, this processing could take a lot of training time. Genomics is a Big Data science with high dimensionality. It relies on deep learning to solve complicated problems in certain diseases like cancer by using different DNA information such as the transcription factor. TAL1 is a transcription factor that is essential for the development of hematopoiesis and of the vascular system. In this paper, we highlight the potential of deep learning in the field of genomics and its challenges such as the training time that takes hours, weeks, and in some cases months. Therefore, we propose to apply a distributed deep learning implementation based on Convolutional Neural Networks (CNN) that showed good results in decreasing the training time and enhancing the accuracy performance with 95% by using multiple GPU and TPU as accelerators. We proved the efficiency of using a distributed strategy based on data-parallelism in predicting the transcription-factor TAL1 motif faster.
As we consider the number of IoT time-sensitive applications , the transfer of data to a remote data center and server such as Cloud, Fog, and Edge becomes inefficient since the deadline constraint is not satisfied. Thus, ensuring that the IoT time-sensitive applications meet their timing constraints is a challenge. Mist Computing is closer to IoT devices, presenting the lowest communication delay but less computational resource than the Cloud, Fog, and Edge. Seeing several IoT devices use MQTT protocol to access the data due to its lightness and flexibility, we propose an architecture for IoT time-sensitive applications based on MQTT protocol and integrating Mist Computing. We focus on distributing the MQTT brokers over Mist nodes to satisfy the deadline constraints with the consideration of the limited resource of Mist nodes. Hence, we propose an approach for the selection of the appropriate MQTT Mist broker. We have also proposed MQTT communication model that provides the M/M/1 based analysis for delay computing and energy conception. The experiment results show that our proposal is very effective for time-sensitive applications and also maximize the lifetime of IoT systems since it minimizes the cumulative energy of the system. Compared to MQTT Edge broker distribution, our solution provides the lesser delay of communication between IoT devices.
IoT is a distributed physical network could support in various services, such as communications integration, control and information-processing via numerous transportation systems. It is an application extends to all aspects of transportation systems such as the vehicle, the infrastructure, and the driver or user. In this paper, we approach integration of Intelligent Traffic Management Systems and IoT from semantic service viewpoint. We create the architecture system of web services based on OWL-S in IoT environment and describe the composition web service based on IoT. We have extended this web services to real-time extensions that can describe the application related to real-time management of the vehicle traffics. To validate our system, we proposed case study for Ambulance Vehicle. This case of study shows the use of the timing and extended real-time features for web service description and the related process ontology generated for OWL-S technology.
Using efficiently the wireless sensor networks based on IEEE 802.15.4/zigbee remains a real challenge for the forest fire detection and monitoring applications. The most relevant question is how we can maintain a long lifetime for the network with the need of fast and active sensor devices for the fire detection. In this paper, we propose a new approach Auto-organization, Adaptive frame Periods for forest Fire detection for multi-level optimization based on the network topology reorganization, and the frame activity period optimization according to the energy preservation and also the fire detection timing constraints. The reorganization is made locally according to the node states with regard to the fire detection events. It is made by a new association/re-association procedure that creates links and paths between nodes with respect to the two constraints. According to the network topology, an adaptive frame periods adjustment procedure is executed to select the suitable timing periods that reduce the sensor node activities without exceeding the timing constraints. The simulation results show superiority and efficiency of the proposed approach for the energy preservation, even if we consider a large network size.
John Domingue合作论文数Knowledge Media Institute
The Open University1