The Wizard of Oz (WOz) technique is an experimental evaluation mechanism. It allows the observation of a user operating an apparently fully functioning system whose missing services are supplemented by a hidden wizard. From our analysis of existing WOz systems, we observe that this technique has primarily been used to study natural language interfaces. With recent advances in interactive media, multimodal user interfaces are becoming popular but our current understanding on how to design such systems is still primitive. In the absence of generalizable theories and models, the WOz technique is an appropriate approach to the identification of sound design solutions. We show how the WOz technique can be extended to the analysis of multimodal interfaces and we formulate a set of requirements for a generic multimodal WOz platform. The Neimo system is presented as an illustration of our early experience in the development of such platforms.

One of the new design goals in Human Computer Interaction is to extend the sensory-motor capabilities of computer systems to better match the natural communication means of human beings. This article proposes a dimension space that should help reasoning about current and future Multi-Sensori-Motor systems (MSM). To do so, we adopt a system centered perspective although we draw upon the “Interacting Cognitive Subsystems” psychological model. Our problem space is comprised of 6 dimensions. The first two dimensions deal with the notion of communication channel: the number and direction of the channels that a particular MSM system supports. The other four dimensions are used to characterize the degree of built-in cognitive sophistication of the system: levels of abstraction, context, fusion/fission, and granularity of concurrency. We illustrate the discussion with examples of multimedia and multimodal systems, both MSM systems but with distinct degrees of built-in cognitive sophistication.

To minimize distractions, a pervasive-computing environment must be context-aware. The authors define an activity-attention framework for context-aware computing, discuss the spatial and temporal aspects of applications they developed, and introduce a pervasive-computing architecture.

Computing devices and applications are now used beyond the desktop, in diverse environments, and this trend toward ubiquitous computing is accelerating. One challenge that remains in this emerging research field is the ability to enhance the behavior of any application by informing it of the context of its use. By context, we refer to any information that characterizes a situation related to the interaction between humans, applications, and the surrounding environment. Context-aware applications promise richer and easier interaction, but the current state of research in this field is still far removed from that vision. This is due to 3 main problems: (a) the notion of context is still ill defined, (b) there is a lack of conceptual models and methods to help drive the design of context-aware applications, and (c) no tools are available to jump-start the development of context-aware applications. In this anchor article, we address these 3 problems in turn. We first define context, identify categories of contextual information, and characterize context-aware application behavior. Though the full impact of context-aware computing requires understanding very subtle and high-level notions of context, we are focusing our efforts on the pieces of context that can be inferred automatically from sensors in a physical environment. We then present a conceptual framework that separates the acquisition and representation of context from the delivery and reaction to context by a context-aware application. We have built a toolkit, the Context Toolkit, that instantiates this conceptual framework and supports the rapid development of a rich space of context-aware applications. We illustrate the usefulness of the conceptual framework by describing a number of context-aware applications that have been prototyped using the Context Toolkit. We also demonstrate how such a framework can support the investigation of important research challenges in the area of context-aware computing.

We present a way of analyzing sensed context information formulated to help in the generation, documentation and assessment of the designs of context-aware applications. Starting with a model of sensed context that accounts for the particular characteristics of sensing, we develop a method for expressing requirements for sensed context information in terms of relevant quality attributes plus properties of the sensors that supply the information. We demonstrate on an example how this approach permits the systematic exploration of the design space of context sensing along dimensions pertinent to software development. Returning to our model of sensed context, we examine how it can be supported by a modular software architecture for context sensing that promotes separation between context sensing, user interaction, and application concerns.

User interfaces must adapt to the growing dissemination of computing power in our everyday environment. Computing devices and applications are now used beyond the desktop, in diverse environments, and this trend is accelerating. By taking context into account, context-aware applications promise richer and easier interaction. However, designers face three major obstacles: (1) the notion of context is still ill-defined; (2) there is a lack of conceptual models and methods to help drive the design of context-aware applications; and (3) no tools are available to jump-start the development of context-aware applications. In this paper, we address these three problems in turn. We first define context, identify categories of contextual information, and characterize context-aware application behavior. We then present a conceptual model of contextaware applications development. This conceptual model introduces abstractions to help deal with context and are implemented in a context toolkit we have built. We propose design rules that help derive relevant context components from the application requirements to build the software architecture of the application. We illustrate our concepts and design rules with a prototype application we have built, a mobile context-aware conference assistant. This example application emphasizes how the abstractions we propose facilitate the description and design of a seemingly complex system. Finally, based on our experience with context-awareness, we outline future challenges for context-aware applications.

Carnegie Mellon University has developed a user-centered interdisciplinary concurrent system design methodology (UICSM) that takes teams of electrical engineers, mechanical engineers, computer scientists, industrial designers and human-computer interaction students that work with an end-user to generate a complete prototype system during a four-month-long course. The methodology is Web-supported and defines intermediary design products that document the evolution of the design. These products are posted on the Web so that even remote designers and end-users can participate in the design activities. The design methodology proceeds through three phases: conceptual design, detailed design and implementation. End-users critique the design at each phase. In addition, simulated and real application tasks provide further focus for design evaluation. The methodology has been used by the class, in designing over a dozen wearable computers, with diverse applications ranging from inspection and maintenance of heavy transportation vehicles to augmented reality in manufacturing and plant operations. The methodology includes monitoring and evaluation of the design process. This methodology is illustrated through a description of developing pervasive computing applications in collaboration with IBM during the Spring 2000 course

User interfaces must adapt to the growing dissemination of computing power in our everyday environment. Computing devices and applications are now used beyond the desktop. Mobile, wearable, and pervasive computing allow users to integrate computing in the flow of their activities in the physical world. But most of our systems are still deaf and blind to anything that isn’t explicitly input by the user. By taking the environmental interaction context into account, context-awareness promises easier interaction and new possibilities for applications. On the surface, there are many similarities between the needs of multimodal and context-aware applications. What can we learn from multimodality to build context-aware systems ? What are the common research problems ? To investigate these issues, we give a brief overview of context-aware systems, including a simple classification. We describe some abstractions we have found useful to help build context-aware applications. We then turn to the main problems facing the developers of context-aware systems and relate them to issues encountered in multimodal systems development.

Context is an important, yet poorly utilized source of information in interactive computing. It is difficult to use because, unlike other forms of user input, there is no common, reusable way to handle context. Most context-aware applications have been built in an ad hoc manner. We discuss the requirements for dealing with context and present an architectural solution we have designed and implemented to help application designers build context-aware applications more easily. We illustrate the use of the architecture through a context-aware application that assists conference attendees.

In order for a smart environment to provide services to its occupants, it must be able to detect its current state or context and determine what actions to take based on the context. We discuss the requirements for dealing with context in a smart environment and present a software infrastructure solution we have designed and implemented to help application designers build intelligent services and applications more easily. We describe the benefits of our infrastructure through applications that we have built.

Context is an important, yet poorly utilized source of information in interactive computing. It is difficult to use because, unlike other forms of user input, there is no common, reusable way to handle context. Most contextaware applications have been built in an ad hoc manner. We discuss the requirements for dealing with context and present an architectural solution we have designed and implemented to help application designers build contextaware applications more easily. We illustrate the use of the architecture through a context-aware application that assists conference attendees.

Context-enabled applications are just emerging and promisericher interaction by taking environmental context into account.However, they are difficult to build due to their distributednature and the use of unconventional sensors. The concepts oftoolkits and widget libraries in graphical user interfaces has beentremendously successtil, allowing programmers to leverage offexisting building blocks to build interactive systems more easily.We introduce the concept of context widgets that mediate betweenthe environment and the application in the same way graphicalwidgets mediate between the user and the application. We illustratethe concept of context widgets with the beginnings of a widgetlibrary we have developed for sensing presence, identity andactivity of people and things. We assess the success of ourapproach with two example context-enabled applications we havebuilt and an existing application to which we have addedcontext-sensing capabilities.

We describe the Conference Assistant, a prototype mobile, context-aware application that assists conference attendees. We discuss the strong relationship between context-awareness and wearable computing and apply this relationship in the Conference Assistant. The application uses a wide variety of contexts and enhances user interactions with both the environment and other users. We describe how the application is used and the context-aware architecture on which it is based.

This paper reports ongoing experience with the design and everyday use of an electronic context-enabled in/out board. We designed this application as part of the development of a context toolkit and it proved a fruitful test-bed for investigating issues of context sensor fusion. We describe the first version of the application that used a single context sensor and explain some usability problems it raised. We analyze the limitations of available context sensors and conclude that the usability problems cannot be overcome using a single sensor. We suggest solutions relying on the use of multiple context sensors and sensor fusion.

Although context awareness is a key component for perceptual user interfaces, we lack generic infrastructure for developing context aware applications. We propose a generic infrastructure based on context servers that store, share and archive contextual data. We describe a few applications we have built that take advantage of context sharing and context history. We then turn to the overall design of our context server and analyze in detail its services with a worked example.

Ubiquitous computing (ubicomp) is an emerging paradigm for interaction between people and computers. A guiding principle of ubicomp is to break away from desktop computing to provide computational services to a user when and where required. Although there has been a lot of experimental work in ubicomp, there has been little effort to define an agenda in ubicomp for HCI researchers. In this paper, we attempt to remedy that problem by defining the space of ubicomp applications in terms of the level of user mobility and transparency of interaction. Increases in user mobility will come with technological advances, but increased interaction transparency will come only with breakthroughs in HCI research. We conclude the paper with a discussion of two functional themes that we have found important across a number of ubicomp systems —context-awareness and automated capture, integration and access. Each of these themes raises special HCI issues and, together with the taxonomy for ubicomp applications, defines a clearer agenda for HCI research in ubiquitous computing.

Information technology is developing at an astounding pace. Computers are now common and their use is switching from number-crunching tools to "information and communication appliances". The growing success of the Internet is just an example of how information and communication facilities are now available to the general public. But today's Internet is just the precursor of more innovative applications. New advances in computer science disciplines such as communications networks and mobile devices are suggesting radically new uses of information technology. These new uses may raise new ethical issues. We give a few examples and show that these issues are not explicitly dealt with in existing ethical frameworks such as Mason's PAPA or Huffs ImpactCS. We also discuss the usability of these frameworks by software designers and suggest that information technology practitioners need more explicit tools such as handbooks to help them understand and deal with ethical issues.

The growth in mobile and distributed computing have highlighted a category of task in which the location of the user and/or resources or other task agents affects the way the task is performed. We identify important features of such tasks and propose requirements for a spatial framework to model location information for task support.