Portalware: A Smartphone-Wearable Dual-Display System for Expanding the Free-Hand Interaction Region in Augmented Reality

Free-hand manipulation in smartphone augmented reality (AR) enables users to directly interact with virtual contents using their hands. However, human hands can ergonomically move in a broader range than a smartphone's field of view (FOV) can capture, requiring users to be aware of the limited usable interaction and viewing regions at all times. We present Portalware, a smartphone-wearable dual-display system that expands the usable interaction region for free-hand manipulation and enables users to receive visual feedback outside the smartphone's view. The wearable is a lightweight, low-cost display that shares the same AR environment in real-time with the smartphone.This setup empowers AR applications such as mid-air drawing and object manipulation by providing a 180-degree horizontal interaction region in front of the user. Other potential applications include wearing the smartphone like a pendant while using Portalware to continue interacting with AR objects. Without having to hold their phone up with their hand, users can benefit from resting their arms as needed. Finally, we discuss usability explorations, potential interactions, and future plans for empirical studies.