Globally, 340 million people have blindness or moderate-to-severe visual impairment (BVI), which limits independent outdoor navigation and negatively affects their health and quality of life. We surveyed 112 people with BVI and found that an ideal outdoor navigation aid must be able to perform turn-by-turn directions, path guidance and obstacle detection and avoidance. Existing navigation tools such as white canes, guide dogs and electronic travel aids often lack one or more of these criteria and may be expensive or inaccessible. Here we introduce Mobilio, a smartphone application that incorporates machine learning, sensor fusion algorithms and personalized audio feedback to meet all of the outdoor navigation criteria. We assessed the reliability of the smartphone sensors and models used for navigation with engineering tests in representative navigation scenarios. We performed a series of experiments in which Mobilio personalized audio feedback for participants with BVI (n = 14), guided them along an outdoor community path and helped them to navigate an obstacle course. Participants walking with Mobilio and a white cane navigated a community path in 13 ± 3% less time and reduced environmental contacts by 41 ± 5% compared with using Google Maps and a white cane. Mobilio achieved similar outdoor navigation reliability to a human guide. Participant surveys reported that Mobilio was easy to use, had a low perceived workload and provided intuitive audio feedback. This work provides an accessible and personalized tool that may be an effective outdoor navigation aid to increase independence for people with BVI.

Background Walking requires managing competing movement goals shaped by the demands of everyday activity. Several important goals during overground walking include maintaining a desired walking speed, achieving an intended foot placement, preserving postural and locomotor balance, and reducing energy expenditure. Biomechanical simulation and motor control-based methods use goal-related cost functions or structured models to identify people's movement priorities but are often restricted to simple movement scenarios and are currently impractical for broad deployment. Methods Here, we systematically vary four movement goals during an overground walking experiment, characterize the resulting trade-offs in spatiotemporal gait metrics, and show that these trade-offs are sufficient to estimate participants' perceived ranking of movement goals. Results Gait metrics across participants became more heterogenous as combinations of movement goal prompts became more complex, suggesting person-specific movement strategies. Participants' perceived prioritization of movement goals showed systematic associations with corresponding gait metrics, with the strongest relationships observed for visually guided goals such as balance and foot placement. Subject-agnostic regression models estimated the perceived order of importance of movement goals with 21% error, and subject-specific models reduced this error to 11%. Conclusions This work establishes an approach for estimation of movement priorities in young, healthy adults using simple gait metrics and provides a framework that could inform future studies beyond the laboratory.

Abstract In "Experiment-free exoskeleton assistance via learning in simulation", Luo et al. [1] present an ambitious framework for developing exoskeleton controllers through reinforcement learning exclusively in computer simulation. The authors report that a control policy trained on a small dataset from one subject was directly transferred to physical hardware, reducing human metabolic cost during walking, running, and stair climbing by more than any prior device. If confirmed, this would represent a major breakthrough for the field of wearable robotics and their clinical applications. However, a close examination of the published materials casts doubt on these claims. The reported experimental results violate physiological limits on the relationship between mechanical power and muscle energy use during gait 2,3,4 . The algorithmic claims are surprising and cannot be verified; in contrast with established replicability standards in machine learning 5,6 , executable code has not been made available. We conclude that the goals of this study have not yet been verifiably achieved and make recommendations for avoiding publication errors of this type in the future.

Smartwatches are commonly used to estimate energy expenditure, but studies have reported errors of 25% to 70%. The large range in estimation accuracy may be due to numerous factors, such as activity type, use case, and sensor data. We evaluated the differences in energy expenditure estimation among different smartwatches and models to identify the optimal use cases to achieve minimal error. We systematically evaluated the impact of different sensor inputs on energy expenditure estimation during outdoor activities by comparing a smartwatch operated in an activity-specific mode that incorporated GPS, IMU, and heart rate data; a smartwatch operated in a generic, non-activity-specific, mode that incorporated IMU and heart rate data; and a pre-defined heart rate model. We compared an activity-specific smartwatch, a non-activity specific smartwatch, and a heart rate model to isolate how different sensor inputs impact energy expenditure estimation. Thirty adults (17 men, 13 women; 34 ± 14 years) completed outdoor activities while wearing a portable respirometry system and both smartwatches. Participants performed self-paced walking and running conditions, and followed ecologically relevant audio prompts to elicit real-world walking. Energy expenditure estimates were collected from each smartwatch, and a predefined heart rate model was used to exclude wrist motion. Cumulative energy expenditure from each model was compared with ground-truth respirometry. The activity-specific smartwatch achieved 13% error during outdoor walking with three times lower error than the non-activity specific smartwatch and heart rate model. During outdoor running, the heart rate model yielded a lower error than both the activity-specific and non-activity specific smartwatches. Energy expenditure estimation reached near laboratory-grade performance when both activity-specific information and GPS data were used for monitoring low-intensity activities, while heart-rate alone appeared better suited for high-intensity activities. Meanwhile, estimates from non-activity specific smartwatches should be interpreted cautiously by researchers, clinicians, and users when monitoring physical activity.

Physical inactivity is the fourth largest cause of global mortality. Health organizations have requested a tool to objectively measure physical activity because many specific and causal relationships between activity and health outcomes are not clearly understood. Existing activity monitors are either unsuitable for large-scale use or have substantial error. We present OpenMetabolics, a biomechanically-informed activity monitor that employs a smartphone in a pants pocket which measures leg motion to estimate energy expenditure. OpenMetabolics uses a data-driven machine learning model to capture the relationship between underlying leg muscle activity and energy expended during common physical activities. OpenMetabolics estimated energy expenditure with 18% cumulative error across all real-world activities, approximately two times lower than existing tools. We developed a pocket motion artifact correction model to accurately monitor energy expenditure when the smartphone is in a pocket of various types of clothing. A week-long, at-home monitoring study highlighted individual and population-level activity patterns across various timescales. We have made the data, code, and smartphone application open source. This accurate and accessible activity monitor could be deployed for large-scale studies with many patient populations to relate activity to health outcomes, inform health policy, and develop interventions.

Smartwatches are commonly used to estimate energy expenditure, but studies have reported errors of 25% to 70%, possibly due to factors such as activity type, use case, and sensor data. We compared three readily available configurations for estimating energy expenditure during outdoor walking and running: a smartwatch operated in its activity-specific mode that incorporated GPS, IMU, and heart rate data; a smartwatch operated in its generic, non-activity-specific mode that incorporated IMU and heart rate data; and a pre-defined heart rate model. Thirty adults (17 men, 13 women; 34 ± 14 years) completed outdoor activities while wearing a portable respirometry system and both smartwatches. Participants performed self-paced walking and running conditions, and followed ecologically relevant audio prompts to elicit real-world walking. Cumulative energy expenditure from each method was compared with ground-truth respirometry. The activity-specific smartwatch achieved 13% error during outdoor walking with 2.2-2.8 times lower error than the generic smartwatch and heart rate model, which could be attributed to its activity-specific algorithm, GPS availability, hardware, or a combination of these factors. The heart rate model yielded a lower error than both the activity-specific and non-activity-specific smartwatches during outdoor running in this cohort. This study reflects practical differences between commercially available setups as device, sensor suite, and proprietary algorithms differ across these tools. Estimates from non-activity-specific smartwatches, such as the Fitbit Charge 4 tested here, should be interpreted cautiously by researchers, clinicians, and users when monitoring physical activity.

From industrial exoskeletons to implantable medical devices, robots that interact closely with people are poised to improve every aspect of our lives. Yet designing these systems is very challenging; humans are incredibly complex and, in many cases, we respond to robotic devices in ways that cannot be modelled or predicted with sufficient accuracy. A new approach, human-in-the-loop optimization, can overcome these challenges by systematically and empirically identifying the device characteristics that result in the best objective performance for a specific user and application. This approach has enabled substantial improvements in human-robot performance in research settings and has the potential to speed development and enhance products. In this Perspective, we describe methods for applying human-in-the-loop optimization to new human-robot interaction problems, addressing each key decision in a variety of contexts. We also identify opportunities to develop new optimization techniques and answer underlying scientific questions. We anticipate that our readers will advance human-in-the-loop optimization and use it to design robotic devices that truly enhance the human experience.

Assistive robotic devices like exoskeletons offer the promise of improving mobility for millions of people. However, developing devices that improve an objective mobility metric is challenging. Human-in-the-loop optimization is a systematic approach for personalizing robotic assistance to maximize a mobility metric that has improved device performance for different metrics and applications. Successfully performing human-in-the-loop optimization requires the experimenter to make many decisions, like selecting the appropriate optimization algorithm, hyperparameters, and convergence criteria. Typically, selecting these experimental settings involves pilot experimentation. We propose an approach that uses a probabilistic surrogate model, mapping assistance parameters to corresponding experimental evaluations of the objective mobility metric, to simulate human-in-the-loop optimization and inform these decisions. In this paper, we form a surrogate model of the metabolic landscape of walking with exoskeleton assistance using an existing experimental dataset. We simulate human-in-the-loop optimization by using a synthetic metabolic landscape model to evaluate the metabolic cost of walking with different assistance parameters, instead of performing an experimental measurement. We perform three simulated scenarios optimizing assistance for an expert subject, a novice subject adapting to the device, and an expert subject with up to 20 assistance parameters. The code and analyses from this work are open-source to promote use by other researchers. Simulation enables direct comparison of optimization settings to inform experimental human-in-the-loop optimization and potentially reduce the resources and time required to develop effective assistive devices.### Competing Interest StatementThe authors have declared no competing interest.

Community-based locomotor training post-stroke has shown improvements in independent ambulation by increasing dose, intensity, and specificity of walking practice. Robotic ankle exoskeletons hold the potential to facilitate continued rehabilitation at home, but understanding what aspects of the design are most relevant for successful translation to the community presents a challenge. Here, we design a portable rigid ankle exoskeleton to use as a research platform for investigating the effect of assistance on post-stroke gait during overground, community-based walking. We first test our device with stroke survivors and validate its potential for future community use. We then present a systematic method for quantifying power transmission losses at each transmission stage from the battery to the wearer, using data gathered from walking trials with healthy participants. Our evaluation method revealed inefficiencies in power transfer at the interface level, likely resulting from the compliance in the structural components of the system, which motivates future redesign considerations. Overall, our method provides a framework to identify and characterize the components that must be redesigned to lower exoskeleton weight and maximize performance.

Personalized exoskeleton assistance provides users with the largest improvements in walking speed 1 and energy economy 2–4 but requires lengthy tests under unnatural laboratory conditions. Here we show that exoskeleton optimization can be performed rapidly and under real-world conditions. We designed a portable ankle exoskeleton based on insights from tests with a versatile laboratory testbed. We developed a data-driven method for optimizing exoskeleton assistance outdoors using wearable sensors and found that it was equally effective as laboratory methods, but identified optimal parameters four times faster. We performed real-world optimization using data collected during many short bouts of walking at varying speeds. Assistance optimized during one hour of naturalistic walking in a public setting increased self-selected speed by 9 ± 4% and reduced the energy used to travel a given distance by 17 ± 5% compared with normal shoes. This assistance reduced metabolic energy consumption by 23 ± 8% when participants walked on a treadmill at a standard speed of 1.5 m s −1 . Human movements encode information that can be used to personalize assistive devices and enhance performance.

Objective: Analyzing human motion is essential for diagnosing movement disorders and guiding rehabilitation for conditions like osteoarthritis, stroke, and Parkinson's disease. Optical motion capture systems are the standard for estimating kinematics, but the equipment is expensive and requires a predefined space. While wearable sensor systems can estimate kinematics in any environment, existing systems are generally less accurate than optical motion capture. Many wearable sensor systems require a computer in close proximity and use proprietary software, limiting experimental reproducibility. Methods: Here, we present OpenSenseRT, an open-source and wearable system that estimates upper and lower extremity kinematics in real time by using inertial measurement units and a portable microcontroller. Results: We compared the OpenSenseRT system to optical motion capture and found an average RMSE of 4.4 degrees across 5 lower-limb joint angles during three minutes of walking and an average RMSE of 5.6 degrees across 8 upper extremity joint angles during a Fugl-Meyer task. The open-source software and hardware are scalable, tracking 1 to 14 body segments, with one sensor per segment. A musculoskeletal model and inverse kinematics solver estimate Kinematics in real-time. The computation frequency depends on the number of tracked segments, but is sufficient for real-time measurement for many tasks of interest; for example, the system can track 7 segments at 30 Hz in real-time. The system uses off-the-shelf parts costing approximately $100 USD plus $20 for each tracked segment. Significance: The OpenSenseRT system is validated against optical motion capture, low-cost, and simple to replicate, enabling movement analysis in clinics, homes, and free-living settings

Physical activity is essential to both physical and mental health, something brought home to many people following sedentary pandemic lockdowns. Even without the lockdowns, many parts of the world have been facing an obesity epidemic, which has created a need to help people manage their weight. For such people there are a wealth of fitness and diet apps that rely on smartphone and smartwatch sensors to monitor activity levels and track the calories they have burned. The problem is that smartphones and smartwatches do a terrible job at calorie counting.

Globally, more than 250 million people have impaired vision and face challenges navigating outside their homes, affecting their independence, mental health, and physical health. Navigating unfamiliar routes is challenging for people with impaired vision because it may require avoiding obstacles, recognizing objects, and wayfinding indoors and outdoors. Existing approaches such as white canes, guide dogs, and electronic travel aids only tackle some of these challenges. Here, we present the Augmented Cane, a white cane with a comprehensive set of sensors and an intuitive feedback method to steer the user, which addresses navigation challenges and improves mobility for people with impaired vision. We compared the Augmented Cane with a white cane by having sighted and visually impaired participants complete navigation challenges while blindfolded: walking along hallways, avoiding obstacles, and following outdoor waypoints. Across all experiments, the Augmented Cane increased the walking speed for participants with impaired vision by 18 ± 7% and sighted participants by 35 ± 12% compared with a white cane. The increase in walking speed may be due to accurate steering assistance, reduced cognitive load, fewer contacts with the environment, and higher participant confidence. We also demonstrate advanced navigation capabilities of the Augmented Cane: indoor wayfinding, recognizing and steering the participant to a key object, and navigating a sequence of indoor and outdoor challenges. The open-source and low-cost design of the Augmented Cane provides a platform that may improve the mobility and quality of life of people with impaired vision.

Physical inactivity is the fourth leading cause of global mortality. Health organizations have requested a tool to objectively measure physical activity. Respirometry and doubly labeled water accurately estimate energy expenditure, but are infeasible for everyday use. Smartwatches are portable, but have significant errors. Existing wearable methods poorly estimate time-varying activity, which comprises 40% of daily steps. Here, we present a Wearable System that estimates metabolic energy expenditure in real-time during common steady-state and time-varying activities with substantially lower error than state-of-the-art methods. We perform experiments to select sensors, collect training data, and validate the Wearable System with new subjects and new conditions for walking, running, stair climbing, and biking. The Wearable System uses inertial measurement units worn on the shank and thigh as they distinguish lower-limb activity better than wrist or trunk kinematics and converge more quickly than physiological signals. When evaluated with a diverse group of new subjects, the Wearable System has a cumulative error of 13% across common activities, significantly less than 42% for a smartwatch and 44% for an activity-specific smartwatch. This approach enables accurate physical activity monitoring which could enable new energy balance systems for weight management or large-scale activity monitoring.

Real-world autonomous systems operate under uncertainty about both their pose and dynamics. Autonomous control systems must simultaneously perform estimation and control tasks to maintain robustness to changing dynamics or modeling errors. However, information gathering actions often conflict with optimal actions for reaching control objectives, requiring a trade-off between exploration and exploitation. The specific problem setting considered here is for discrete-time nonlinear systems, with process noise, input-constraints, and parameter uncertainty. This article frames this problem as a Bayes-adaptive Markov decision process and solves it online using Monte Carlo tree search with an unscented Kalman filter to account for process noise and parameter uncertainty. This method is compared with certainty equivalent model predictive control and a tree search method that approximates the QMDP solution, providing insight into when information gathering is useful. Discrete time simulations characterize performance over a range of process noise and bounds on unknown parameters. An offline optimization method is used to select the Monte Carlo tree search parameters without hand-tuning. In lieu of recursive feasibility guarantees, a probabilistic bounding heuristic is offered that increases the probability of keeping the state within a desired region.

This paper presents Stanford Doggo, a quasi-direct-drive quadruped capable of dynamic locomotion. This robot matches or exceeds common performance metrics of state-of-the-art legged robots. In terms of vertical jumping agility, a measure of average vertical speed, Stanford Doggo matches the best performing animal and surpasses the previous best robot by 22%. An overall design architecture is presented with focus on our quasi-direct-drive design methodology. The hardware and software to replicate this robot is open-source, requires only hand tools for manufacturing and assembly, and costs less than $3000.

Background Estimating energy expenditure with indirect calorimetry requires expensive equipment and several minutes of data collection for each condition of interest. While several methods estimate energy expenditure using correlation to data from wearable sensors, such as heart rate monitors or accelerometers, their accuracy has not been evaluated for activity conditions or subjects not included in the correlation process. The goal of our study was to develop data-driven models to estimate energy expenditure at intervals of approximately one second and demonstrate their ability to predict energetic cost for new conditions and subjects. Model inputs were muscle activity and vertical ground reaction forces, which are measurable by wearable electromyography electrodes and pressure sensing insoles. Methods We developed models that estimated energy expenditure while walking (1) with ankle exoskeleton assistance and (2) while carrying various loads and walking on inclines. Estimates were made each gait cycle or four second interval. We evaluated the performance of the models for three use cases. The first estimated energy expenditure (in Watts) during walking conditions for subjects with some subject specific training data available. The second estimated all conditions in the dataset for a new subject not included in the training data. The third estimated new conditions for a new subject. Results The mean absolute percent errors in estimated energy expenditure during assisted walking conditions were 4.4%, 8.0%, and 8.1% for the three use cases, respectively. The average errors in energy expenditure estimation during inclined and loaded walking conditions were 6.1%, 9.7%, and 11.7% for the three use cases. For models not using subject-specific data, we evaluated the ability to order the magnitude of energy expenditure across conditions. The average percentage of correctly ordered conditions was 63% for assisted walking and 87% for incline and loaded walking. Conclusions We have determined the accuracy of estimating energy expenditure with data-driven models that rely on ground reaction forces and muscle activity for three use cases. For experimental use cases where the accuracy of a data-driven model is sufficient and similar training data is available, standard indirect calorimetry could be replaced. The models, code, and datasets are provided for reproduction and extension of our results.

Robots performing manipulation tasks must operate under uncertainty about both their pose and the dynamics of the system.In order to remain robust to modeling error and shifts in payload dynamics, agents must simultaneously perform estimation and control tasks.However, the optimal estimation actions are often not the optimal actions for accomplishing the control tasks, and thus agents trade between exploration and exploitation.This work frames the problem as a Bayesadaptive Markov decision process and solves it online using Monte Carlo tree search and an extended Kalman filter to handle Gaussian process noise and parameter uncertainty in a continuous space.MCTS selects control actions to reduce model uncertainty and reach the goal state nearly optimally.Certainty equivalent model predictive control is used as a benchmark to compare performance in simulations with varying process noise and parameter uncertainty.

The benefits of legged locomotion shown in nature overcome challenges such as obstacles or terrain smoothness typically encountered with wheeled vehicles. This paper evaluates the benefits of using optimal control on a single leg hopper during the entire hopping motion. Basic control without considering physical constraints is implemented through hand-tuned PD controllers following the Raibert control framework. The differential flatness of the first-order equations of motion and the Liouvillian property for the second-order equations for the hopper system are proved, enabling flat outputs for control. A two-point boundary value problem (BVP) is then used to minimize jerk in the flat system to gain implicit smoothness in the output controls. This smoothness ensures that the planned trajectories are feasible, allowing for given waypoints to be reached.

Surgeries involving interaction with soft tissue like the brain need to minimize shear and normal forces that can cause tissue damage or hemorrhage. Other surgeries require the ability to follow a complex, curved path, such as through an intestine or to a kidney stone. This paper presents a soft catheter that has the potential to aid in these challenging cases. The soft catheter is capable of apical extension in which the tip extends while the rest of the catheter remains stationary. This limits shear forces with the environment, easing movement of a body's tip through a constrained space. The soft catheter is pre-formed to patient-specific trajectories, meaning that normal forces against tissue would only arise due to errors between the actual and desired paths; we show decrease in normal force applied to the environment on the order of 100 compared to a standard catheter in a 30 degree bend. Setting the internal pressure allows for control of catheter stiffness, with a 500 times difference over the range of tested pressures. Manual operation to reach a surgical site requires only holding the correct orientation at the entry point into the body and setting the internal pressure of the catheter. This soft catheter could offer two benefits: the ability to apply low tissue interaction forces and reach challenging locations within the body.