Distributed storage systems, built on peer-to-peer networks, can provide large-scale data storage and high data reliability by redundancy. Data backup is the process to store data into a set of redundant storage nodes. Rapid completion of such a process is very critical to maintain system performance. In traditional data backup in distributed systems based on erasure codes, star-structured scheme is used, in which each redundant block is just sent to each target storage node from the source node directly, so the storage throughput and delay are limited by the bottleneck bandwidth, due to bandwidth heterogeneity. The recent in-network redundancy generation scheme uses locally repairable property of self-repairing codes to speed up data backup. However, such kind of code does not own maximum distance separable property, thus does not achieve optimal storage efficiency. We still lack a fast backup scheme in distributed systems based on general erasure coding. To this end, we proposed that instead of only focusing on bandwidths between the source node and target nodes, the bandwidths between target storage nodes should be fully taken into account. In our scheme, each redundant data block is divided into some parts according to different proportions and each part of the block is sent to the target storage node via other different storage nodes. The benefit is that spare bandwidths between target storage nodes are used to reduce backup time. We further show how this process can be modeled and derive a formula about the final backup time. We can achieve minimum backup time by solution for classical quadratic programming problem. We conduct both numerical analysis and experimental study. Our experiments shows, the delay reduces 59%, compared with common star-structured scheme. Meanwhile, the throughput is increased significantly in backup process.

Optimizing the Map process is important for the improvement of the MapReduce performance. Many efforts have been devoted into the problem to design more efficient scheduling strategies. However, there exists a kind of MapReduce applications, named imprecise applications, where the imprecise results based on part of map tasks can satisfy the requirements of imprecise applications and thus the job processes can be completed when enough map tasks are processed. According to the feature of imprecise applications, the authors propose an improved MapReduce model, named MapCheckReduce, which can terminate the map process when the requirements of an imprecise application is satisfied. Compared to MapReduce, a Check mechanism and a set of extended programming interfaces are added to MapCheckReduce. The Check mechanism receives and analyzes messages submitted by completed map tasks and then determines whether to terminate the map phase according to the analysis results. The programming interfaces are used by the programmers to define the termination conditions of the map process. A data-prefetching mechanism is designed and implemented in MapCheckReduce which can improve the performance of MapCheckReduce effectively. The MapCheckReduce prototype has been implemented and experiment results verify the feasibility and effectiveness of MapCheckReduce.

Quality of service QoS optimization for end-to-end e2e services always depends on performance analysis in cloud-based service delivery industry. However, performance analysis of e2e services becomes difficult as the scale and complexity of virtualized computing environments increase. In this paper, the authors present a novel hierarchical stochastic approach to evaluate the QoS of e2e virtualized cloud services using Quasi-Birth Death structures, where jobs arrive according to a stochastic process and request virtual machines VMs, which are specified in terms of resources, i.e., VM-configuration. To reduce the complexity of performance evaluation, the overall virtualized cloud services are partitioned into three sub-hierarchies. The authors analyze each individual sub-hierarchy using stochastic queueing approach. Thus, the key performance metrics of e2e cloud service QoS, such as acceptance probability and e2e response delay incurred on user requests, are obtained.

Distributed storage systems can provide large-scale data storage and high data reliability by redundant schemes, such as replica and erasure codes. Redundant data may get lost due to frequent node failures in the system. The lost data is needed to be regenerated as soon as possible so as to maintain data availability and reliability. The direct way for reducing regeneration time is to reduce network traffic in regeneration. Compared with that way, tree-structured regeneration achieves shorter regeneration time by constructing better tree-structured topology to increase transmission bandwidth. However, some bandwidth of many other edges beyond the tree is not utilized to speed up transmission in tree-structured regeneration. In this paper, we consider to use multiple edge-disjoint trees to parallel regenerate the lost data, and analyze the total regeneration time. We deduce the formula about optimal regeneration time, and propose an approximate construction algorithm with polynomial time complexity for the optimal multiple regeneration trees. Our experiments shows, the regeneration time reduces 62

To provide timely results for 'Big Data Analytics', it is crucial to satisfy deadline requirements for MapReduce jobs in production environments. In this paper, we propose a deadline-oriented task scheduling approach, named Dart, to meet the given deadline and maximize the input size if only part of the dataset can be processed before the time limit. Dart uses an iterative estimation method which is based on both historical data and job running status to precisely estimate the real-time job completion time. By comparing the estimated time with the deadline constraint, a YARN-based task scheduler dynamically decides whether continuing or terminating the map phase. We have validated our approach using workloads from OpenCloud and Facebook on a cluster of 60 virtual machines. The results show that Dart can not only effectively meet the deadline but also process near-maximal data volumes even when the deadline is set to be extremely small and limited resources are allocated.

Cloud storage system provides reliable service to users by widely deploying redundancy schemes in its system – which brings high reliability to the data storage, but inversely introduces significant overhead to the system, consisting of storage cost and energy consumption. The core behind this issue is how to leverage the relationship between data redundancy and data reliability. To optimize both concurrently is apparently difficult. As such, to fix one as a constraint and then to reach another one becomes the consensus. We aim in the paper to pursue a storage allocation scheme that minimizes the data redundancy while achieving a given (high) data reliability. For this purpose, we have provided a novel model based on generating function. With this model, we have proposed a practical and efficient storage allocation scheme, which is proved to be able to minimize the data redundancy. We analytically demonstrate that the suggested solution brings several advantages, in particular the reduction of the search space and the acceleration to the computation. We also assess the improvement on the savings of data redundancy experimentally by adopting availability traces collected from real world – which encouragingly shows that the reduction of data redundancy by our solution can reach up to more than 30% as compared to the heuristic method recently proposed in the research community.

The scheduling approach in MapReduce may result in the "long tail" problem because of the unreasonable task assignment and high scheduling overhead because of an amount of task scheduling operations. To address these problems, a new task scheduling approach for MapReduce, named "Iterative Task Scheduling Algorithm", is proposed. The new approach tries to schedule the map tasks according to the solution of the equation for the optimal task assignment. Thus the "long tail" problem can be mitigated effectively and the task scheduling operations can be significantly reduced. To support our new scheduling approach, two approaches are proposed: The first one is adopted to estimate task execution times of nodes and the second one is adopted to produce the optimal task assignment based on the known task execution times of nodes. Comprehensive experiments have been performed with the real log data from the Ali Cloud and the results verify the effectiveness of the new task scheduling approach. The map runtime of the job is reduced 23% in our experiments.

In distributed storage systems, erasure codes represent an attractive data redundancy solution which can provide the same reliability as replication requiring much less storage space. Multiple data losses happens usually and the lost data should be regenerated to maintain data redundancy in distributed storage systems. Regeneration for multiple data losses is expected to be finished as soon as possible, because the regeneration time can influence the data reliability and availability of distributed storage systems. However, multiple data losses is usually regenerated by regenerating single data loss one by one, which brings high entire regeneration time and severely reduces the data reliability and availability of distributed storage systems. In this paper, we propose a tree-structured parallel regeneration scheme based on regenerating codes (TPRORC) for multiple data losses in distributed storage systems. In our scheme, multiple regeneration trees based on regenerating code are constructed. Firstly, these trees are created independently, each of which dose not share any edges from the others and is responsible for one data loss; secondly, every regeneration tree based on regenerating codes owns the least network traffic and bandwidth optimized-paths for regenerating its data loss. Thus it can perform parallel regeneration for multiple data losses by using multiple optimized topology trees, in which network bandwidth is utilized efficiently and entire regeneration is overlapped. Our simulation results show that the tree-structured parallel regeneration scheme reduces the regeneration time significantly, compared to other regular regeneration schemes.

The massive data in Data centers network will be frequently accessed massive datasets for cloud services, which will lead to some new requirements and becomes an important issue for interconnection topology and data management in cloud computing. According to the cost-effective, the paper proposes a new interconnection network MyHeawood for cloud computing. MyHeawood is constructed by small switches and servers with dual-port NIC according to recursive method. The data placement strategy in MyHeawood is a hashing algorithm based on the family of hash functions. MyHeawood uses three replicas strategy base on master copy, which is allocated in different sub layer to improve the reliability of data.

Parallel computing can improve the data-processing efficiency significantly. However, the traditional approaches, such as MPI and MapReduce, need to program in the special environment. In this paper, a new distributed computing framework named MEX is proposed. Users just provides the input files and the name of an executable program to MEX. Then MEX will automatically process these files on a cluster of machines with the executable program. The MEX platform has been designed and implemented based on MapReduce and some key problems are addressed. An improved map function are designed for the start-up of the executable program. To support the improved map function, a data-conversion mechanism is added into MEX which generates the command texts as the parameter of the map function. A process-feedback mechanism is proposed for the fault-tolerance of the executable program. The mechanism also supports the synchronous execution between the map task and the executable program, which can avoid too many processes to be started on the same worknode. Comprehensive experiments are performed to verify the effectiveness of the MEX framework. According to the results, more computing worknodes can result in less job runtime in MEX. When 100 virtual machines are used for an OCR job with 1000 images in 400 dpi, the runtime is reduced 88.6% compared to a single machine.

Erasure code is an efficient redundancy solution to avoid permanent data losses in distributed storage system. By using erasure code, the original data is encoded into blocks which are stored on different nodes. When one node fails, the lost data block needs to be recovered by transmitting a large number of data from helper nodes, which is called repair. Intersecting zigzag sets codes (IZS codes) and hierarchical code are two kinds of codes which reduce the bandwidth overhead in repair. IZS codes reduce the amount of data uploaded by each helper node, while hierarchical codes reduce the number of nodes participating in repair. In this paper, we propose a class of code called intersecting zigzag sets-hierarchical code (IZ2SHC code) to unify them together. IZ2SHC code decreases repair degree as hierarchical code and reduces repair overhead compared with the latter. Besides, it realizes exact regeneration of the failed data, and need not encode in helper node during the repair, which reduces disk read cost for helper node when transferring data in repair.

Due to high storage efficiency, erasure codes are recently used to provide high data reliability in distributed storage systems. When multiple data loses in system, regeneration time for them demands to be as short as possible so as to keep data availbility and reliability. Common way is to repair them one by one, which prolongs the regeneration time. Tree-structured regeneration may reduce regeneration time when regenerating one single node failure by relaying the network traffic, and is also extended to regenerate multiple data losses. In this paper, based on regenerating codes which achieve minimal network traffic during the regeneration, we consider reducing regeneration time by using multiple max-min trees to parallel regenerate multiple data losses. And we proposed an algorithm: bandwidth-sharing max-min algorithm (BSM2RC) to construct multiple parallel max-min trees. It realizes efficient bandwidth utilization by maximizing the minimal bottleneck edge weight of multiple regeneration trees, thus improve regeneration efficiency. Our simulation experiment shows that multiple parallel max-min trees reduce total regeneration time for multiple data losses significantly, and thus enhance system reliability, compared with existing regeneration scheme.

Optimizing the Map Phase is an important way to reduce the MapReduce (MR) job runtime. The common way for such studies is to design more efficient scheduling policies. However, there exists a kind of MapReduce applications, named imprecise applications, where the reduce phase can be completed based on the outputs of part of the map tasks. According to the feature of imprecise applications, we propose an improved MapReduce model, named MapCheckReduce (MCR). MCR can terminate the map process when the requirements of imprecise applications are satisfied. Compared to MR, a Check mechanism and a set of extended programming interfaces are added to MCR. The Check mechanism can receive and analyze messages submitted by mappers and then determine whether to terminate the map phase. The programming interfaces can be used by the programmers of imprecise applications to define the termination conditions of the map phase. The MCR prototype has been implemented and experiment results verify the feasibility and effectiveness of MCR.

Sensor information system is a specific distributed information management system for applying sensor data and aims to effectively process, manage, and analyze data emanating from sensor networks. Recently, with the development of sensor networks, sensor information system encounters many challenges, such as huge and diverse data, heterogeneous clients, scalability, and security. In this paper, we propose an extensible and secure cloud architecture model for sensor information system. Firstly, we describe the composition and mechanism of the architecture model using cloud paradigm. Secondly, we design the security solution for accessing sensor data and information services inside the architecture. This security solution ensures legal access and use for sensor data and information services and avoids illegal breach for user data in the cloud environment. In particular, a certificate authority (CA) based Kerberos protocol is proposed to provide strong identity authentication. At last, we introduce the experiment deployment for the architecture. This architecture fully utilizes the virtualization technology and cloud service functions and builds an extensible cloud platform environment supplying scalable, flexible, reliable, and secure sensor information services, which aims to solve the challenges encountered by the present sensor information system. The simulation experiment results show that the architecture gets high performance and stability throughout, while keeping scalability and flexibility brought by cloud.

Wireless sensor networks (WSN) is a key technology extensively applied in many fields, such as transportation, health-care and environment monitoring. Despite rapid development, the exponentially increasing data emanating from WSN is not efficiently stored and used. Besides, the data from multiple different types and locations of WSN needs to be well analyzed, fused and supplied to various types of clients, such as PC, workstation and smart phone. The emerging cloud computing technology provides scalable data process and storage power and some types of connectable services, which can helpfully utilize sensor data from WSN. In this paper, we propose an integration framework of cloud computing with WSN, in which sensor data is transmitted from WSN to cloud, and processed and stored in cloud, then mined and analyzed so as to be supplied to various clients. By applying virtualization and cloud storage technology, and Infrastructure as a Service (IaaS) and Software as a Service (SaaS) of cloud service model, the framework can fully process and store mass sensor data from multiple types of WSN. Besides, it efficiently mines and analyzes sensor data, based on which the data applications are well supplied to various types of clients in form of services.

Wireless sensor networks (WSN) is a critical technology for information gathering covering many areas, including health-care, transportation, air traffic control and environment monitoring. Despite wide use, the fast increasing data emanating from WSN is not fully utilized due to the limitation for structure of WSN itself. Along with the further development of WSN, the data form which is not be efficiently managed and applied to supply information services for users. As the emerging IT technology, cloud computing supplies powerful utilization ability for IT resources, which makes many traditional applications migrate to cloud computing. In this paper, we propose a framework integrating cloud computing paradigm and WSN, which fully uses data process ability and service model for cloud computing. In the framework, data form WSN are efficiently utilized and managed, depending on which, information services for WSN are well provided to users.

Recently, Cloud computing, as one of the hottest words in IT world, has drawn great attention. Many IT companies such as IBM, Google, Amazon, Microsoft, Yahoo and others vigorously develop cloud computing systems and related products to customers. However, there are still some difficulties for customers to adopt cloud computing, in which many security issues exist, because data for a customer is stored and processed in cloud, not in a local machine. This paper briefly introduces cloud computing and its key concepts. In particularly, we intend to discuss security requirements and security issues involving data, application and virtualization in cloud computing, as well as current solutions to these issues.

Although multi-touch technology is currently a research focus in the field of Human-Computer Interaction, its relative research, however, is still comparatively few in China. In this paper, several foreign multi-touch technologies based on senor and computer vision are introduced and the advantages and disadvantages of these technologies are analyzed briefly. It is important for studying the technology of detection and tracking touch-point in multi-touch. Furthermore the FTIR (Frustrated Total Internal Reflection) and DI (Diffused Illumination) which are based on computer vision multi-touch technology are highlighted. Finally, several crucial techniques in the field of multi-touch technology are also discussed.