A parallel sort-merge-join algorithm that uses a divide-and-conquer approach to address the data skew problem is proposed. The algorithm adds an extra scheduling phase to the usual sort, transfer and join phases. During the scheduling phase, a parallelizable optimization algorithm, using the output of the sort phase, attempts to balance the load across the multiple processors in the subsequent join phase. The algorithm naturally identifies the largest skew elements and assigns each of them to an optimal number of processors. Assuming a Zipf-like distribution for data skew, the algorithm is shown to achieve very good load balancing for the join phase in a CPU-bound environment and to be very robust relative to the degree of data skew and the total number of processors.< >
An extended version of the SIMD (single-instruction, multiple-data) relational algebraic processor is presented. In addition to the usual relational and set operations the machine has the ability to recycle its responder sets internally. This allows it to perform repeated joins, for example, without external intervention and so achieve operations such as path discovery and transitive closure in graphs stored as relations and to evaluate various types of recursive query. The many compiled methods for recursive query evaluation are applicable in this system, as in any other relational database, and can be efficiently evaluated because of the in-built recursive and iterative capability of the machine. The magic functions approach has a clear connection with the machine, since it uses relations as magic functions
The issues and solutions relation to intra-query parallelism in a relational DBMS (database management systems) supporting SQL are addressed. A broad framework for the study of the numerous issues that need to be addressed in supporting parallelism efficiently and flexibly is provided. The impact that parallelization of complex queries has on short transactions which have stringent response time constraints is discussed. The advantages and disadvantages of the shared nothing, shared disks and shared everything architectures for parallelism are enumerated. The impact of parallelism on a number of components of an industrial-strength DBMS are pointed out. The different stages of query processing during which parallelism may be gainfully used are identified. The interactions between parallelism and the traditional systems' pipelining technique are analyzed. Finally, the performance implications of parallelizing a specific complex query are studied. This gives a range of sample points for different parameters of a parallel system architecture, namely I/O and communication bandwidth as a function of aggregate MIPS (million instruction per second).<>
A prototype of an object-oriented system implemented in CProlog is described. Its main objective is to demonstrate system features that would support efficient management of objects and object-oriented databases in a persistent and distributed environment. Mechanisms at the low level of the system were considered to support object distribution, mobility control and configuration management in a simple and uniform way. Objects exist in clusters which are transparent to the applications. The prototype is a framework for a self-organizing object-oriented distributed system
Text retrieval systems which store extremely large amounts of text while providing a multiuser retrieval service for a large customer base are discussed. Due to the severe I/O demands of such a system, it is usually beneficial, if not necessary to utilize a multiprocessor system with multiple I/O facilities in an effort to increase the parallel I/O activity, the objective being to lower search response times. The problem is defined, a solution is modeled and it is shown that the application can be handled in a very effective fashion by a multiprocessor system with a simple LAN-based topology. A type of functional splitting which, if done in a careful manner, helps improve search response time is described.<>
A mixed MIMD/SIMD execution model for a reconfigurable computer is presented. This model is adapted to the use of a specialized associative coprocessor embedded in this host machine. A main characteristic of the model is that it uses four types of processes (decoding, calculus, coprocessor communication and transaction manager) and that in principle one process of each type is allowed on each processor. Time intervals are allocated to operations into partitions of the set of processors. Transfers are usually limited to identifiers, logical addresses and locks. Simulations display a high level of processor occupation. Therefore the machine yield may be very high, and the operations should be fast
Given a set of functional dependencies Sigma and a single dependency sigma , it is shown that the algorithm to test whether Sigma implies sigma is log-space complete in P. The functional dependencies Sigma are represented as a directed hypergraph H/sub Sigma /. A parallel algorithm is presented which solves the above implication problem using P processors on an exclusive-read, exclusive-write parallel random access machine (EREW-PRAM) in O(e/P+n log P) time and on a concurrent-read, concurrent-write PRAM (CRCW-PRAM) in O(e/P+n) time, where e and n are the number of arcs and nodes of the graph H/sub Sigma /. For graphs H/sub Sigma / with fixed degree and diameter, it is shown that the closure H/sub Sigma //sup +/ can be computed in NC. NC algorithms are presented to obtain a nonredundant and a LR-minimum cover for the set of functional dependencies Sigma . All the algorithms on an n-node directed hypergraph with fixed degree and diameter can be implemented to run in O(log/sup 2/ n) time with M(n) processors on a CREW-PRAM model, where M(n) is the cost of multiplying two binary matrices. The algorithms are efficient based on the transitive closure bottleneck phenomenon.< >
A low overhead delay method for deciding when to correct a strategy is presented. Sampling is used to estimate the size relations and alternative heuristic strategies prepared in a background mode are used to decide when to correct. Correction is made only if lower overall delay is achieved, including correction time. Evaluation using a model of a distributed database indicates that the heuristic strategies are near optimal. Moreover, it suggests that it is usually correct to abort creation of an intermediate relation which is much larger than predicted
Hierarchical concurrency control has been proposed as one possible approach for multidatabase systems. However, to apply this approach, some restrictions must be imposed on the local concurrency control algorithms. The restriction is identified. Based on this restriction, the hierarchical concurrency control approach is formalized and its correctness is proven. A novel global concurrency control algorithm based on this hierarchical approach is also presented.< >
The performance of a functional disk system with relational database engine (FDS-RII) for a nonuniform data distribution is analyzed. FDS-RII is a relational storage system, designed to accelerate relational algebraic operations which uses a hash-based algorithm to process relational operations. In the has-based algorithm, a relation is first partitioned into several clusters by a split function. Each cluster is then staged onto the main memory and further, a hash function is applied to each cluster to perform a relational operation. Thus, the nonuniformity of split and hash functions is considered to result from a nonuniform data distribution on the hash-based algorithm. It is possible to attenuate the effect of the hash function nonuniformity by increasing the number of processors and processing the buckets in parallel. In order to address the nonuniformity of split function, the combined hash algorithm is introduced. This algorithm combines the grace hash algorithm with the nested loop algorithm in order to handle the overflown bucket efficiently. Using the combined hash algorithm, it is found that the execution time of the nonuniform data distribution is almost equal to that of the uniform data distribution
A preliminary evaluation of some multijoin strategies and their performances on parallel hardware is described. The hardware used was a Sequent (under Unix) with 11 usable processors, each with shared and private primary memory. A multijoin was broken down into a series of single joins which were then allocated to clusters, each cluster being a collection of parallel processors. The results of single joins, which were studied by both binary search and hash-merge techniques, were then further processed as necessary. The evaluation was conducted varying a number of parameters, such as cluster size, tuple size and cardinality. The comparative results were plotted. The study highlights the importance of a number of factors that influence the performance of a multijoin operation.<>
An approach to the design of replica control schemes which can provide higher availability than currently existing voting schemes is proposed. The approach is based on the observation that the existing voting schemes actually belong to a general class, called voting class. This class of voting schemes can be represented in a very simple and uniform way. Thus, a designer can choose the optimal scheme within the voting class by evaluating each of them and choosing the one which maximizes the availability.<>
The run-time performance of a strategy (previously proposed by the authors, J. Shao et al., 1989) for processing recursive queries in deductive database system is studied. The algorithm, introduced informally by examples, is coded in occam2 and runs on a network of transputers. A wide range of recursive queries and database structures are used as benchmarks. Both the speedup factors achieved and the elapsed time spent by the strategy in answering recursive queries are analyzed. Experimental results show that it is possible to achieve significant performance improvements when queries are evaluated in parallel. These results provide insights into the success of this strategy in meeting the primary objective of focusing on relevant data.<>
Semijoin has traditionally been relied upon for reducing the communication cost required for distributed query processing. However, judiciously applying join operations as reducers can lead to further reduction in the communication cost. In view of this fact, the approach of using join operations, in addition to semijoins, as reducers in distributed query processing is explored. It is shown that the problem of determining a sequence of join operations for a query graph can be transformed to that of finding a set of cuts to that graph, where a cut to a graph is a partition of the nodes in that graph. In light of the mapping, an efficient heuristic algorithm to determine an effective sequence of join reducers for a query is developed. The algorithm, using the concept of divide-and-conquer, is shown to have polynomial time complexity. Examples are given to illustrate the results
A model of distributed data processing is introduced and discussed. For this purpose, a typical application system is analyzed and divided into subapplications. To fulfil the task of the global application, the subapplications must communicate in an appropriate manner by exchanging data. In the proposed model the communication between subapplications is split up into two steps: the offering of information by sending subapplications and its acceptance by receiving subapplications. For both communication steps, synchronous and asynchronous processing modes are defined. Supporting those different communication modes, the cooperation between subapplications can be defined very closely to the specific demands of the application system. This optimizes distributed data processing. The prototype implementation of a distributed data management system, based on the flexible communication mechanism described is demonstrated.<>
The JOYCE+ model for multisite distributed systems is presented and the operational semantic of the asynchronous communication between processes is illustrated with Petri nets. The syntax of the JOYCE+ language is presented in terms of the Guarded Commands language. The expressive power of the JOYCE+ language in distributed synchronization problems with timeout handling is illustrated through examples. Software development environments (based on JOYCE+) for distributed systems over multi- and monoprocess computer networks are discussed.< >
Complex objects to support non-standard database applications require the use of substantial computing resources because their powerful operations and their related integrity constraints must be performed and maintained in an interactive environment. Since the exploitation of parallelism within such operations seems to be promising, we investigate the principal approaches for processing a query on complex objects (molecules) in parallel. A number of arguments favor methods based on inter-molecule parallelism as against intramolecule parallelism. Retrieval of molecules may be optimized by multiple storage structures and access paths. Hence, maintenance of such storage redundancy seems to be another good application area to explore the use of parallelism. Deferred update seems to be a bad idea, whereas concurrent update strategies incorporate salient application features. For performance reasons, we have chosen a multiprocessor system sharing an instruction addressable common memory which is used for buffer management, synchronization, and logging/recovery. Activation of concurrent requests is supported by a nested transaction concept which allows a safe and effective execution control within parallel actions of an operation.
A novel, high performance subsystem for information retrieval called JAS is introduced. The JAS subsystem sustains gigabyte per second comparison rates due to its “early-out” CMOS substring search processor. The complexity of each JAS unit is independent of the complexity of the query. This query complexity independence is achieved via the utilization of a decoupled instruction set. The JAS subsystem architecture is described and the performance is evaluated.
We propose a method of parallelizing bottom-up-evaluation of logic programs. The method does not introduce interprocess communication, or synchronization overhead. We demonstrate that it can be applied when evaluating several classes of logic programs, e.g., the class of linear single rule programs. This extends the work reported in [WS] by significantly expanding the classes of logic programs that can be evaluated in parallel. We also prove that there are classes of programs to which the parallelization method cannot be applied.
