For nearly ten years my group and I at Thinking Machines Corporation worked at selling massively parallel computers for a variety of applications that fall broadly in the area now called “database mining.” We had an amazing team of scientists and engineers, saw trends far ahead of the rest of the world, and developed several great systems. However, we began as novices in the business arena. Sometimes we made sales, sometimes we did not; but we learned a great deal in either case. This chapter recounts the sales process and a brief history, mostly in the form of “war stories” mixed with technical details, and attempts to summarize some messages to take away, based on what we learned.
This work presents how Weighted Inheritance Networks, a new formalism that is capable of representing knowledge under a nonmonotonic multiple inheritance scheme with exceptions, support the design of massively parallel algorithms for problems related to nonmonotonic reasoning. Weighted Inheritance Networks are semantic networks with links that are annotated with symbolic pairs that we call weights. They can handle the inheritance structures described in the literature, including some which the so far proposed formalisms may fail to treat satisfactorily. Weighted Inheritance Networks support the development of both sequential and parallel efficient algorithms, without sacrificing any desired properties that inheritance networks should, in general, possess. We investigate the performance of parallel nonmonotonic multiple inheritance reasoners based on Weighted Inheritance Networks, for two important problems: the goal-directed inheritance reasoning problem, that consists in answering whether an object possesses a particular property and the recognition problem, that consists in finding all the objects possesing a particular set of properties. We present a system that provides massively parallel support for nonmonotonic “path-based” reasoning, called WINBRID, and we analyze its performance using randomly generated knowledge bases.
In this paper we describe a parallel implementation of object-level propagation in the Description-Logic (DL) system FLEX. We begin by analyzing the parallel potential of the main DL inference algorithms normalization, subsumption checking, classification, and object-level propagation. Instead of relying on a parallelism inherent in logic programming languages, we propose to exploit the application-specific potentials of DLs and to use a more data-oriented parallelization strategy that is also applicable to imperative programming languages. Propagation turns out to be the most promising inference component for such a parallelization. We present two alternative PROLOG implementations of parallelized propagation on a loosely coupled MIMD (Multiple Instruction, Multiple Data) system, one based on a farm strategy, the other based on distributed objects. Evaluation based on benchmarks containing artificial examples shows that the farm strategy yields only poor results. The implementation based on distributed objects, on the other hand, achieves a considerable speed-up, in particular for large-size applications. We finally discuss the impact of these results for real applications.
Any intelligent system capable of common sense reasoning and language understanding must be capable of performing rapid inferences with reference to a large body of knowledge. The ability to perform rapid inferences with large knowledge bases is also essential for supporting flexible and effective access to the enormous body of electronically available data. Since complexity theory tells that not all inferences can be computed effectively, it is important to identify interesting classes of inference that can be performed effectively. Over the past several years we have tried to do so by working within a neurally motivated, massively parallel computational model. Our approach is motivated by the belief that situating the knowledge representation and reasoning problem within a neurally motivated computational architecture will not only enhance our understanding of the mind/brain, but it will also lead to the development of effective knowledge representation and reasoning systems implemented on existing hardware. In this chapter we substantiate this claim and review some results of pursuing this approach. These include a characterization of reflexive reasoning—reasoning that can be performed effectively by neurally plausible networks; the design of CSN—a connectionist semantic network that can perform inheritance and recognition in time proportional to the depth of the conceptual hierarchy; shruti—a connectionist knowledge representation and inference system that can encode a large number of facts, rules, and a type hierarchy, and perform a class of first-order inferences with extreme efficiency, and shruti-cm5 an implementation of shruti on the CM-5 that can encode over half a million rules, facts, and types and respond to reflexive queries within a few hundred milliseconds.
This chapter discusses the performance evaluation of low-level computer vision (CV) algorithms operating in parallel and distributed environments. In low-level CV algorithms, most of the operations are local the new value associated with each pixel depends only on the values coming from a well-defined and limited neighborhood of that pixel. Therefore, low-level CV problems are suited to be ported on a parallel or distributed environment as they show also the most balanced behavior from the point of view of the computation versus communication ratio. The cellular neural network (CNN) paradigm is appropriate to describe this kind of computation as it embodies , as special cases, all CV problems solved with algorithms involving local operations. Hence, the use of CNNs to evaluate the performance of CV applications in parallel environments is the best choice as CNNs are both a superset of all local low-level CV algorithms and suited for parallelization. A measurement taken with respect to the most general CNN formulation can give an effective value of the lower performance bound offered by a given platform for CV applications.
We have ported the SIMD Parka knowledge representation system to generic MIMD machines. The system has been recoded in C and supported using runtime optimization packages developed in the High Performance Systems Software Laboratory at the University of Maryland. New “scanning” algorithms have been developed for inheritance and recognition inferences. These algorithms have been tested with both random networks and on a recoding of the ontology of the CYC knowledge base as well as on large planning case-bases. Tests show that the new version is significantly faster than the SIMD system, and that it promises to scale well to knowledge bases orders of magnitude larger than CYC.
This paper continues a series of papers dealing with the problems of (1) fast verification of the existence of a transitive relation in an IS-A hierarchy, and (2) dynamic update of such a hierarchy. As in our previous work, a directed acyclic graph (DAG) of IS-A relationships is replaced by a set of nodes, annotated by number pairs, and stored on a massively parallel computer. In this paper a new mapping of this set of nodes onto the processors is described, called the Double Strand Representation (DSR). The DSR improves the processor usage compared to our previously used Grid Representation (GR). This paper shows IS-A verification and number pair propagation algorithms for the Double Strand Representation. Test runs on a CM-5 Connection Machine 3 are reported.
This paper presents a decomposition-based distributed algorithm for solving constraint satisfaction problems. The main alternatives for distributed constraint satisfaction are reviewed. An algorithm using a partition of the constraint graph is then detailed, with its parallel version. Experiments on problems made of loosely connected random constraint satisfaction problems show its benefits for under-constrained problems and problems with a complexity in the phase transition zone.
We present an alternative mechanism for representing concurrent theorem-proving activity which primarily relies upon massive parallelism and efficient broadcast communication. This model of distributed deduction can be utilised in order to provide more dynamic, flexible and open systems. In addition to the representation of deduction in classical propositional and first-order logics framework, we provide correctness results for the approach, and consider the practical aspects of the system's implementation. The approach to concurrent theorem-proving we propose is based upon the use of asynchronously executing concurrent objects. Each object contains a particular set of formulae and broadcasts messages corresponding to specific information about those for-mulae. Based upon the messages that on object receives, it can make simple inferences, transforming the formulae it contains and sending out further meassages as a result. Communication can be organised is such a way that, if a formula, distributed across a set of objects, is unsatisfiable then at least one object will eventually derive a contradiction. In addition to representing simple deduction in this framework, we indicate how, by utilising the underlying computational model, information not directly associated with the formulae being manipulated may be encoded. In particular, complex control behaviour can be provided, allowing not only the implementation of a range of proof strategies, including opportinistic, competitive and cooperative deduction, but also providing the basis for the development of simple agents societies.
In this paper, we present several kinds of programs developed in a new parallel language, ParCeL-1. This language is based on autonomous actors that compute concurrently as virtual processors. The applications we present here cover various domains of interest to Artificial Intelligence, especially tree search and connectionist programming. We present general methods to develop such kinds of algorithms in ParCeL-1. Then we emphasize several rules that should be followed to write efficient parallel programs. Finally, we describe the performances of these applications on two parallel computers.
Many artificial intelligence techniques and applications rely on performing heuristic search through large problem spaces. Iterative-Deepening-A* (IDA*) search has proven to be effective for large search spaces, because it requires no intermediate state storage and is guaranteed to find optimal solutions. However, the time taken to perform IDA* search on real-world tasks often prevents the everyday usage of AI techniques. Parallel processing can considerably reduce the time spent in search, and can thereby speed up AI applications. This paper describes HyPS, a hybrid parallel window/distributed tree search algorithm. Using this algorithm, the set of available processors is divided into clusters. Each cluster searches simultaneously through the same search space, but to a unique cost threshold. Within each cluster, the search space is divided so that an individual processor will search a fraction of the total search space. Operator ordering and load balancing techniques are used to further improve the performance of HyPS. Results on two real-world and one artificial domains show a substantial performance improvement over serial search algorithms, and indicate an improvement over existing parallel search approaches. In this paper we also describe a mechanism for automatically selecting the optimal number of clusters to use.
A framework for induction of decision trees suitable for implementation on shared- and distributed-memory multiprocessors or networks of workstations is described. The approach, called Parallel Decision Trees (PDT), overcomes limitations of equivalent serial algorithms that have been reported by several researchers, and enables the use of the very-large-scale training sets that are increasingly of interest in real-world applications of machine learning and data mining.
Static Partitioning with Slackness (SPS) is a method for parallelizing search-based systems. Traditional partitioning approaches for parallel search rely on a continuous distribution of search alternatives among processors (“dynamic partitioning”). The SPS-model instead proposes to start with a sequential search phase, in which tasks for parallel processing are generated. These tasks are then distributed and executed in parallel. No partitioning occurs during the parallel execution phase. The potentially arising load imbalance can be controlled by an excess number of tasks (slackness) as well as appropriate task generation. The SPS-model has several advantages over dynamic partitioning schemes. The most important advantage is that the amount of communication is strictly bounded and minimal. This results in the smallest possible dependence on communication latency, and makes efficient execution even on large workstation networks feasible. Furthermore, the availability of all tasks prior to their distribution allows optimization of the task set not possible otherwise. The paper describes the basic SPS-model, presents general simulation results, provides a worst-case comparison with other parallelization approaches, and discusses the appropriateness of using the model for parallelization.
A voxelization of a mathematically defined continuous object in Euclidean 3-space is a set of voxels that may be used as a discrete representation of the continuous object. The generation of voxelizations is an important part of volume graphics. This chapter deals with the voxelization of surfaces. In this case the voxelization should be a thin connected set of voxels with appropriate local separation properties. The voxelization should be free of small holes that would allow a discrete ray in its complement to pass from one side of the continuous surface to the other. No part of the continuous surface should be far from the voxelization, and no part of the voxelization should be far from the surface. We state several mathematical conditions which express these requirements in a precise way. Some of the conditions were proposed a number of years ago (in a rather different context) by Morgenthaler and Rosenfeld. Two voxelizations of planes in 3-space are described, and shown to satisfy all of the conditions.
The general problem of shrinking binary images is addressed with emphasis on the problem of shrinking to a residue. Past work in this area is reviewed, fundamental limits are discussed and open questions are identified. Emphasis is given to techniques which can be used to verify correct performance of shrinking algorithms, including successful algorithm termination and connectivity preservation. New connectivity preservation tests are developed for parallel 2D reductive-augmentative algorithms and the application of these tests is demonstrated for a variety of shrinking algorithms. A new 2D completely parallel single-operator shrinking algorithm is developed using connectivity preservation requirements to guide the design process. Issues in 3D shrinking are also reviewed.
This text for graduate students discusses the mathematical foundations of statistical inference for building three-dimensional models from image and sensor data that contain noise--a task involving autonomous robots guided by video cameras and sensors. The text employs a theoretical accuracy for the optimization procedure, which maximizes the reliability of estimations based on noise data. The numerous mathematical prerequisites for developing the theories are explained systematically in separate chapters. These methods range from linear algebra, optimization, and geometry to a detailed statistical theory of geometric patterns, fitting estimates, and model selection. In addition, examples drawn from both synthetic and real data demonstrate the insufficiencies of conventional procedures and the improvements in accuracy that result from the use of optimal methods.
This chapter develops basic topics concerning the skeletonization of planar patterns in binary pictures digitized on the square grid, and focuses on some aspects of existing skeletonization methods designed for implementation on conventional sequential computers. More details on iterative, non-iterative, and other methods, as well as a comprehensive list of references, can be found in [1,2].
In machine vision, an original gray tone image is processed to produce features that can be used by higher-level processes, such as recognition and inspection procedures. Thresholding the image results in a binary image whose pixels are labeled as foreground or background. Segmenting the image results in a symbolic image whose pixels are assigned labels representing various classifications. In both cases, an important next step in the analysis of the image is an operation called connected component labeling that groups the pixels into regions, such that adjacent pixels have the same label, and pixels belonging to distinct regions have different labels. Properties of the regions and relationships among them may then be calculated. The most common relationship, spatial adjacency, can be represented by a region adjacency graph. This chapter describes algorithms for connected component labeling and region adjacency graph construction. In addition to giving several sequential algorithms for two-dimensional connected component labeling, it also discusses several parallel algorithms and an algorithm for three-dimensional connected component labeling.
Intuitively, a boundary in an N-dimensional digital space is a connected component of the (N 1)-dimensional surface of a connected object. In this paper we make these concepts precise, and show that the boundaries so specified have properties that are intuitively desirable. We provide some efficient algorithms for tracking such boundaries. We illustrate that the algorithms can be used, in particular, for computer graphic display of internal structures (such as the skull and the spine) in the human body based on the output of medical imaging devices (such as CT scanners). In the process some interesting mathematical results are proven regarding "digital Jordan boundaries," such as a specification of a local condition that guarantees the global condition of "Jordanness."
