For the Internet of Things to move from the lab to the real world, software and application development must be simplified, and collaboration must increase. We designed an operating system and a development environment that facilitate the process and supports a range of users.

As architects usually decide on the shape and look of windows during the design of buildings, opportunities for interactive windows have not been systematically explored yet. In this work, we extend the vision of sustainable and comfortable adaptive buildings using interactive smart windows. We systematically explore the design space of interactive windows to chart requirements, constraints, and challenges. To that end, we built proof-of-concept prototypes of smart windows with fine-grained control of transparency. In two studies, we explored user attitudes towards interactive windows and elicited control methods. We found that users understand and see potential for interactive windows at home. We provide specific usage contexts and specify interactions that may facilitate domestic applications. Our work illustrates the concept of interactive smart windows and provides insights regarding their design, development, and user controls for adaptive walls. We identify design dimensions and challenges to stimulate further development in the domain of adaptive buildings.

As architects usually decide on the shape and look of windows during the design of buildings, opportunities for interactive windows have not been systematically explored yet. In this work, we extend the vision of sustainable and comfortable adaptive buildings using interactive smart windows. We systematically explore the design space of interactive windows to chart requirements, constraints, and challenges. To that end, we built proof-of-concept prototypes of smart windows with fine-grained control of transparency. In two studies, we explored user attitudes towards interactive windows and elicited control methods. We found that users understand and see potential for interactive windows at home. We provide specific usage contexts and specify interactions that may facilitate domestic applications. Our work illustrates the concept of interactive smart windows and provides insights regarding their design, development, and user controls for adaptive walls. We identify design dimensions and challenges to stimulate further development in the domain of adaptive buildings.

Touchscreens are the most successful input method for smartphones. Despite their flexibility, touch input is limited to the location of taps and gestures. We present PalmTouch, an additional input modality that differentiates between touches of fingers and the palm. Touching the display with the palm can be a natural gesture since moving the thumb towards the device's top edge implicitly places the palm on the touchscreen. We present different use cases for PalmTouch, including the use as a shortcut and for improving reachability. To evaluate these use cases, we have developed a model that differentiates between finger and palm touch with an accuracy of 99.53 % in realistic scenarios. Results of the evaluation show that participants perceive the input modality as intuitive and natural to perform. Moreover, they appreciate PalmTouch as an easy and fast solution to address the reachability issue during one-handed smartphone interaction compared to thumb stretching or grip changes.

Previous research and recent smartphone development presented a wide range of input controls beyond the touchscreen. Fingerprint scanners, silent switches, and Back-of-Device (BoD) touch panels offer additional ways to perform input. However, with the increasing amount of input controls on the device, unintentional input or limited reachability can hinder interaction. In a one-handed scenario, we conducted a study to investigate the areas that can be reached without losing grip stability (comfortable area), and with stretched fingers (maximum range) using four different phone sizes. We describe the characteristics of the comfortable area and maximum range for different phone sizes and derive four design implications for the placement of input controls to support one-handed BoD and edge interaction. Amongst others, we show that the index and middle finger are the most suited fingers for BoD interaction and that the grip shifts towards the top edge with increasing phone sizes.

Exploring large geolocated social media datasets is now an important task in many pursuits e.g. crisis response. Yet there is still a lack of effective methods to view and interact with large amounts spatially-disturbed user-generated content. In this work, we explore interaction techniques for an extended version of ScatterBlogs - an interactive application for exploring massive twitter datasets on large high-resolution displays. We designed an interaction technique that employs multiple tablets to enable multiple users to effectively manipulate geolocated twitter massages on a large screen. In a preliminary user study, we compared our technique with using a desktop computer. Results indicate that the technique offers superior performance and user experience. In future work, we will explore how our technique can enhance the user experience of interacting with analytics applications.

Touchscreens are successful in recent smartphones due to a combination of input and output in a single interface. Despite their advantages, touch input still suffers from common limitations such as the fat-finger problem. To address these limitations, prior work proposed a variety of interaction techniques based on input sensors beyond the touchscreen. These were evaluated from a technical perspective. In contrast, we envision a smartphone that senses touch input on the whole device. Through interviews with experienced interaction designers, we elicited interaction methods to address touch input limitations from a different perspective. In this work, we focus on the interview results and present a smartphone prototype which senses touch input on the whole device. It has dimensions similar to regular phones and can be used to evaluate presented findings under realistic conditions in future work.

Window facades play an increasingly important role in modern architecture. Regular shutters and blinds allow only coarse control over the sunlight coming through windows. Smart windows using see-through displays can be controlled on a per-pixel basis and thereby have the potential of fine-grained control. In this paper, we explore future interaction with such smart windows and conducted an elicitation study with 16 potential users. We provide both a mid-air gesture set and a smartphone interface to define regions for glare protection and brightness control. The study was conducted on a working 1.6 x 2.6 m smart window prototype with 130 x 144 individually switchable pixels.

Smartphones, wearables, and other mobile devices often use tactile feedback for notifying users. This feedback type proved to be beneficial since it does not occupy the visual or auditory channel. However, it still can be distracting in other situations such as when users are already stressed. To investigate tactile feedback patterns which do not increase the user's stress level, we developed two wrist-worn prototypes capable of providing tactile feedback (i.e., vibrotactile and pressure-based feedback). Further, we conducted a user-study with 14 participants comparing both feedback types. The results suggest that vibrotactile feedback increases the user's stress level more, compared to pressure-based feedback particularly applied when the user currently has a low stress level. Consequently, we present implications for designing notifications for mobile and wearable devices.

Previous research proposed a wide range of interaction methods and use cases based on the previously unused back side and edge of a smartphone. Common approaches to implementing Back-of-Device ( BoD) interaction include attaching two smartphones back to back and building a prototype completely from scratch. Changes in the device's form factor can influence hand grip and input performance as shown in previous work. Further, the lack of an established operating system and SDK requires more effort to implement novel interaction methods. In this work, we present a smartphone prototype that runs Android and has a form factor nearly identical to an off-the-shelf smartphone. It further provides capacitive images of the hand holding the device for use cases such as grip-pattern recognition. We describe technical details and share source files so that others can re-build our prototype. We evaluated the prototype with 8 participants to demonstrate the data that can be retrieved for an exemplary grip classification.

With increasingly large smartphones, it becomes more difficult to use these devices one-handed. Due to a large touchscreen, users can not reach across the whole screen using their thumb. In this paper, we investigate approaches to move the screen content in order to increase the reachability during one-handed use of large smartphones. In a first study, we compare three approaches based on back-of-device (BoD) interaction to move the screen content. We compare the most preferred BoD approach with direct touch on the front and Apple's Reachability feature. We show that direct touch enables faster target selection than the other approaches but does not allow to interact with large parts of the screen. While Reachability is faster compared to a BoD screen shift method, only the BoD approach makes the whole front screen accessible.

With the Internet of Things (IoT), we are now building a network of more and more connected household devices which share information and react to each others and users. However, little progress has been made on interaction with buildings themselves. In particular, smart components are attached to windows, such as window blinds or shutters, but the window itself has not changed much. Most components we use on a daily basis are operated mechanically and controlled manually. This results in costly repairs due to wear out and severe weather conditions especially in large buildings. In this work, we show how interaction could change in the future with smart windows based on display technology. We present a self-contained and connected smart window prototype which can block sunlight on a per pixel basis. Further, we propose different input methods and discuss the opportunities of implicit as well as explicit input. This is a first step towards making the digital revolution an architectural one with IoT technology.

Our eyes use multiple cues to perceive depth. Current 3D displays do not support all depth cues humans can perceive. While they support binocular disparity and convergence, no commercially available 3D display supports focus cues. To use them requires accommodation, i.e. stretching the eye lens when focusing on an individual distance. Previous work proposed multilayer and light field displays that require the eye to accommodate. Such displays enable the user to focus on different depths and blur out content that is out of focus. Thereby, they might ease the separation of content displayed on different depth layers. In this paper we investigate the effect of focus cues by comparing 3D shutter glasses with a multilayer display. We show that recognizing content displayed on a multilayer display takes less time and results in fewer errors compared to shutter glasses. We further show that separating overlapping content on multilayer displays again takes less time, results in fewer errors, and is less demanding. Hence, we argue that multilayer displays are superior to standard 3D displays if layered 3D content is displayed, and they have the potential to extend the design space of standard GUI.

Today, most digital devices are either stationary, often placed on horizontal surfaces like tables, or socalled mobile devices which are carried around by the user. In this demonstration we showcase our ongoing work on a novel type of self-actuated display. It can be placed on walls, or whiteboards and other arbitrarily oriented surfaces like ceilings. It is equipped with a whiteboard marker which allows the device to draw on surfaces it is attached to. In this work, we demonstrate the device's capabilities using an interactive scenario in which users are able to remotely control the self-actuated display to draw lines on a whiteboard. They control the device either by using their own smart phone or a provided tablet computer.

This contribution presents a mobile modular low-cost open source 3D printed eye tracking prototype, equipped with two off-the-shelf webcams. We compared the accuracy with two state-of-the-art commercial remote eye trackers. In order to verify the benefit of head stabilization, the devices have been tested with and without a chin rest. Experiments have been conducted to determine the feasibility of an open source system and the field of application for this kind of eye tracking device in the low-cost price segment. This opens new use cases and possibilities for indoor and outdoor usability studies and offers new opportunities to apply eye tracking on mobile devices.

Most current devices are passive regarding their locations by being integrated in the environment or require to be carried when used in mobile scenarios. In this paper we present a novel type of self-actuated devices, which can be placed on vertical surfaces like whiteboards or walls. This enables vertical tangible interaction as well as the device interacting with the user through self-actuated movements. In this paper, we explore the application space for such devices by aggregating user-defined application ideas gathered in focus groups. Moreover, we implement and evaluate four interaction scenarios, discuss their usability and identify promising future use cases and improvements.

Touch screens became truly pervasive through the success of smartphones and tablet PCs. Several approaches to further improve the interaction with touch screens have been proposed. In this paper we combine and extend two of these trends. We present a mobile 3D screen that consists of a stack of displays and is touch sensitive on both display sides. This design makes the screen independent from the user's view angle. Using a touch-sensitive back enables back-of-device interaction to avoid the fat-finger problem. Combining back-of-device interaction with a transparent display also avoids occlusion of the user's finger on the back through the device. Through a study we investigate how back and front touch improves interaction with 3D content and show how back-of-device interaction is improved if the user can actually see the finger on the back.

See-through has been simulated for handheld devices to enable back-of-device interaction and for augmented reality applications. Researchers explored a wide range of applications for such devices and technologies to realize them. In this paper we revise previous work on simulated and real see-through handheld devices and their applications. Based on previous work and our own experience with see-through devices, we argue that using pseudo-transparency has inherent drawbacks. We discuss that usable transparent handheld devices require adaptive transparency, consideration of binocular disparity, and new ways to capture content. Furthermore, we indicate approaches to address these factors.

Touch screens are currently the dominant technology for facilitating input and output for mobile devices. Several directions to extend the possibilities of current touch screen technologies have been explored. In this demonstration we showcase a handheld device that consists of a stack of three see-through displays. Using three display layers enables to realize a volumetric 3D display. As our device is touch sensitive on both display sides it enables touch input on the device's front and back. We demonstrate the device's capabilities through three demo applications. We present 3D images using three display layers, demonstrate a game where the character can move from layer to layer, and show a target selection task to compare selection performance on different display layers.