The purpose of this paper is to provide an overview of sonification research, including the current status of the field and a proposed research agenda. This paper was prepared by an interdisciplinary group of researchers gathered at the request of the National Science Foundation in the fall of 1997 in association with the International Conference on Auditory Display (ICAD).

The world we hear is completely different from the world we experience through other sensory modalities. There are many features of the auditory world that make sound an excellent means of conveying information to the user of a system. Sound may be used in traditional alarms, in process monitoring displays, in computer system interfaces, and even to present complex multivariate data. It should be made clear that we are talking about the intentional (as opposed to incidental) use of sound. That is, sounds that are designed as part of the user interface of a system, as opposed to sounds that are unintentionally part of a system, but still convey information to a listener (e.g. the syncopated rhythm of an engine that indicates it is misfiring). Furthermore, for the most part we are considering only nonspeech audio.

This special issue of ACM Transactions on Applied Perception is intended to commemorate the tenth International Conference on Auditory Display (ICAD) and to serve as an introduction and overview of the field of auditory displays. This paper discusses the goals of the issue and describes the paper selection process. The selected papers are also introduced, with their connections to each other, their place in ICAD, and their relevance to other fields briefly highlighted.

The original Walker and Kramer paper at ICAD 1996 studied the mapping of data dimensions (e.g., temperature) onto sound dimensions (e.g., pitch). In this commentary we consider the historical and methodological context of that early work, discuss its relevance to the field of auditory display, and point out how it forms part of a body of work with ties to other researchers in the ICAD community.

Auditory displays are becoming more and more common, but there are still no general guidelines for mapping data dimensions (e.g., temperature) onto display dimensions (e.g., pitch). This paper presents experimental research on different mappings and metaphors, in a generic process-control task environment, with reaction time and accuracy as dependent measures. It is hoped that this area of investigation will lead to the development of mapping guidelines applicable to auditory displays in a wide range of task domains.

The goal of task scheduling in a multiprocessor system is to schedule dependent tasks on processors such that the processing time is minimized. This ensures optimal usage of the processing systems. However this problem is NP-hard in nature and heuristic based techniques are used to obtain a good schedule in polynomial time. Genetic Algorithms (GA) have been proposed over other heuristics because it can use its genetic processes to find multiple solutions faster. The GA proposed is based on a non-preemptive precedence relation between tasks in the task graph. Tasks assignment is prioritized based on the number of tasks dependencies (NTD) and the earliest start time (EST) of each task. For tasks with multiple possible earliest start times, the minimum earliest start time is chosen for such tasks. Java simulations compared the results obtained using the minimum EST and the maximum EST. Our simulation shows that the proposed algorithm with minimum EST achieves faster processing periods compared with the maximum EST. Keywords—Genetic Algorithm, Number of Task Dependencies, Total Finishing Time, Multiprocessor Scheduling.

Many auditory displays use acoustic attributes such as frequency, intensity, and spectral content to represent different characteristics of multidimensional data. This study demonstrated a perceptual interaction between dynamic changes in pitch and loudness, as well as perceived asymmetries in directional acoustic change, that distorted the data relations represented in an auditory display. Three experiments showed that changes in loudness can influence pitch change, that changes in pitch can influence loudness change, and that increases in acoustic intensity are judged to change more than equivalent decreases. Within a sonification of stock market data, these characteristics created perceptual distortions in the data set. The results suggest that great care should be exercised when using lower level acoustic dimensions to represent multidimensional data.

We determined preferred data-to-display mappings by asking experiment participants directly and then examined the psychophysical scaling functions relating perceived data values to underlying acoustic parameters. Presently, we are extending and validating the scaled mappings in practical data interpretation tasks. The resulting scaling functions, in conjunction with the experimental paradigm developed here, should spark further research in this area and have implications for the design of future sonifications.

Many sonification techniques use acoustic attributes such as frequency, intensity, and timbre to represent different characteristics of multidimensional data. Here we demonstrate a perceptual interaction between changes in pitch and loudness, as well as perceived asymmetries in directional change. Three experiments show that changes in loudness can influence judgments of pitch change, changes in pitch can influence loudness change, and that increases in loudness are judged to change more than equivalent decreases. Within a sonification of stock market data, these characteristics created perceptual distortions in the data set. The results imply that in situations where precision is critical, caution should be exercised when using lower level acoustic dimensions such as frequency and intensity to represent multidimensional data.

The idea behind sonification is that synthetic nonverbal sounds can represent numerical data and provide support for information processing activities of many differentkinds. This article describes some of the ways that sonification has been used in assistive technologies, remote collaboration, engineering analyses, scientific visualisations, emergency services and aircraft cockpits. Approaches for designing sonifications are surveyed, and issues raised by the existing approaches and applications are outlined. Relationsare drawn to other areas of knowledge where similar issueshave also arisen, such as human-computer interaction, scientific visualisation, and computer music. At the end is a listof resources that will help you delve further into the topic.

Acoustic designers face new challenges as audio, video and computers converge into the synergy of multimedia. The sound engineer must consider the human factors and acoustic ecology involved in the new "conversation" between listeners and a multimedia system. An organized and centralized sound design will make the audio element a rich ingredient in the multimedia experience. Human Factors and the Acoustic Ecology: Considerations for Multimedia Audio Design. Bruce N. Walker Psychology Department, Rice University Gregory Kramer Clarity/Santa Fe Institute Acoustic designers face new challenges as audio, video and computers converge into the synergy of multimedia. The sound engineer must consider the human factors and acoustic ecology involved in the new "conversation" between listeners and a multimedia system. An organized and centralized sound design will make the audio element a rich ingredient in the multimedia experience. 0 INTRODUCTION: INTERACTIVE AUDIO Audio has typically been part of the recreational and entertainment worlds, while computers have mostly been information retrieval tools. There have been some notable exceptions (Speeth, 1961), but sound was generally used only as a warning, and often redundant with a visual display (Patterson, 1982; Kramer, 1994a). Visual displays have been a huge part of computing for decades, so there is a well-established display style. Audio, on the other hand, is really only beginning to play a significant role in consumer computing. Until very recently, the few guidelines that did exist for auditory displays focused on issues related to warnings, such as audible thresholds and masking (Patterson, 1982; Sorkin, 1987). Now, however, audio designers face new and interesting challenges as audio, video and computers converge into the synergy of multimedia. The listener now is an active part of the system; the designer must consider many issues that have not been as important in other, more traditional, audio realms. While perhaps new to some audio engineers, these ideas come from the well-established field of human factors. In particular, three issues to consider are: the nature of the interaction; the user audience; and the resulting acoustic ecology. 1 THE INTERACTIVE LISTENER The sound engineer must accept a "conversational model" of cooperative interaction between the user and the system (Shneiderman, 1992). This is quite different from the traditional "concert model" of sound presentation, which treats the listener as a passive recipient of the audio presentation. The multimedia listener pursues the dual goals of being entertained and of gaining information. While entertainment may or may not involve interaction, information retrieval and efficient learning is usually interactive, so the listener has evolved into an active player in the audiovisual exchange. This means that the display system, and therefore the sound designer, must be sensitive to what the user wants from the system at any given time. Of course, in a multimedia context, this balance of information and entertainment will change dynamically, providing a challenge for the audio design. But just as in any conversation, communication norms will emerge. For example, when the aim is mostly entertainment and the application is not a game, the user will generally watch and listen to the resulting presentation in a fairly passive way, similar to the way one traditionally watches a movie on video. The computer can take more control over the presentation, and present an active sound environment with many simultaneous audio events that add to the richness of the experience. This is the time for the full orchestra all MULTIMEDIA HUMAN FACTORS the bells and whistles, as it were. Note that the listener will not appreciate even well-intended interruptions from the system. The caveat to this mode of interaction is that it may be very short-lived in the multimedia environment, as new information sources are chosen, or as the user moves on to a new task. On the other hand, if the aim is to extract specific information from any particular audio source, be it recorded speech, a musical selection, a data sonification, or an audible system warning, other sound sources should not interfere. In this situation, the user will play a more active role and will expect the system to respond in an active way. This mode of interaction is more dynamic, and the system needs to respond in an appropriate and timely fashion. For the audio designer, this means keeping the sound sources focused on the information task at hand. Simplify the audioscape and give control of it back to the user. The system needs to seek and provide feedback in this style of conversation. Remember that the listener is still the leader of the conversation and this may result in a non-linear, non-sequential path through the sound and information space. The designer must respect the roles of the participants in the dialog, considering their goals, pace, need for privacy and the willfulness of the user. The user must maintain the feeling of control over the audio environment (and the interface in general); the multimedia system is simply an integrated tool to help achieve the listener's goals for the session.