Vision-Language-Action (VLA) policies have shown promising manipulation capabilities, yet their practical impact is often limited by the reliability demands of real-world deployment. We present a deployment study of an industrial packaging task at Siemens Factory (GWE, Erlangen, Germany), where a robot must pick a transparent accessory bag from a cluttered pile, insert it into the remaining cavity of a cardboard package, and ensure that the bag and its contents remain below the closing plane. Our goal is to understand the practical effort required to adapt a pretrained Pi0.5 policy to a single factory-floor task through iterative fine-tuning and deployment-driven refinement. The pipeline consists of repeated loops of data collection, curation, fine-tuning, evaluation, and targeted recovery data collection. We have accumulated 2535 episodes (10 hours) from the on-site factory settings. In this paper, we contribute an empirical account of a factory-floor VLA deployment, highlighting recurring failure modes and lessons that inform how to improve the deployment workflow.

While reinforcement learning (RL) allows generalist robot policies to continually improve during deployment, the large model size of modern generalist policies, such as VLAs, poses a fundamental obstacle to effective RL improvement. In particular, their severe inference latency—which can lead to pauses or jerky movements—can alter the effective environment dynamics and, if not correctly accounted for, break the Markov assumption that RL relies on, causing standard RL algorithms to fail completely. In this work, we introduce a latency-aware framework, Asynchronous RL with Intermediate Information (ARLI), that enables RL-based improvement of generalist policies under inference delays. Our framework builds on asynchronous inference approaches, which interleave action generation with execution to hide latency, and addresses its incompatibility with RL by providing a low-latency RL policy design that maximizes reactivity within the inference window through two contributions: state augmentations that restore near-Markovian structure by incorporating committed actions and a mid-inference observation. We evaluate our approach across simulated and real-world manipulation tasks, and find that it enables effective finetuning under inference delays where standard RL fails entirely, even matching or exceeding the performance of standard RL in idealized no-latency settings.

While vision-language-action (VLA) models have shown great promise for robot manipulation, their deployment on rigid industrial robots remains challenging due to the inherent trade-off between compliance and responsiveness. Standard Behavior Cloning (BC) approaches predict discrete poses at low frequencies, omitting the velocity and acceleration feedforward terms typically used by low-level compliant controllers. This requires to rely on high stiffness for accurate tracking, thereby sacrificing safe contact dynamics. In this paper, we demonstrate the importance of integrating velocity feedforward terms into VLA policies to resolve this trade-off. We propose two methods for extracting velocity targets from VLAs: a time-discrete finite-difference approximation that serves as a highly effective bridge for existing models, and a continuous Cubic B-Spline action space that natively yields C^2 continuous trajectories for high-frequency control. Crucially, both approaches are strictly model-agnostic and compatible with any standard action-chunking architecture, requiring modifications only to teleoperation, data processing, and the low-level controller. We fine-tune the π_0.5 model and evaluate both of our approaches on a demanding, contact-rich cube-in-hole task. Our results indicate that incorporating the velocity feedforward term via finite differences significantly improves task execution speed, while the continuous B-Spline approach maintains high overall success rates and provides a foundation for smoother higher-order derivatives without compromising compliance.

We prove a noncommutative real Nullstellensatz for 2-step nilpotent Lie algebras that extends the classical, commutative real Nullstellensatz as follows: Instead of the real polynomial algebra R[X1, ... , X d ] we consider the universal enveloping *- algebra of a 2-step nilpotent real Lie algebra (i.e. the universal enveloping algebra of its complexification with the canonical *-involution). Evaluation at points of Rd is then generalized to evaluation through integrable *-representations, which in this case are equivalent to filtered *-algebra morphisms from the universal enveloping *-algebra to a Weyl algebra. Our Nullstellensatz characterizes the common kernels of a set of such *-algebra morphisms as the real ideals of the universal enveloping *-algebra. (c) 2024 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC license (http:// creativecommons.org/licenses/by-nc/4.0/).

We develop a general theory of symmetry reduction of states on (possibly non-commutative) *-algebras that are equipped with a Poisson bracket and a Hamiltonian action of a commutative Lie algebra $g$. The key idea advocated for in this article is that the ``correct'' notion of positivity on a *-algebra $A$ is not necessarily the algebraic one, for which positive elements are sums of Hermitian squares $a^*a$ with $a \in A$, but can be a more general one that depends on the example at hand, like pointwise positivity on *-algebras of functions or positivity in a representation as operators. The notion of states (normalized positive Hermitian linear functionals) on $A$ thus depends on this choice of positivity on $A$, and the notion of positivity on the reduced algebra $A_{red}$ should be such that states on $A_{red}$ are obtained as reductions of certain states on $A$. We discuss three examples in detail: Reduction of the *-algebra of smooth functions on a Poisson manifold $M$, reduction of the Weyl algebra with respect to translation symmetry, and reduction of the polynomial algebra with respect to a $U(1)$-action.

We study formal and non-formal deformation quantizations of a family of manifolds that can be obtained by phase space reduction from [Formula: see text] with the Wick star product in arbitrary signature. Two special cases of such manifolds are the complex projective space [Formula: see text] and the complex hyperbolic disc [Formula: see text]. We generalize several older results to this setting: The construction of formal star products and their explicit description by bidifferential operators, the existence of a convergent subalgebra of “polynomial” functions, and its completion to an algebra of certain analytic functions that allow an easy characterization via their holomorphic extensions. Moreover, we find an isomorphism between the non-formal deformation quantizations for different signatures, linking, e.g., the star products on [Formula: see text] and [Formula: see text]. More precisely, we describe an isomorphism between the (polynomial or analytic) function algebras that is compatible with Poisson brackets and the convergent star products. This isomorphism is essentially given by Wick rotation, i.e. holomorphic extension of analytic functions and restriction to a new domain. It is not compatible with the [Formula: see text]-involution of pointwise complex conjugation.

To economically deploy robotic manipulators the programming and execution of robot motions must be swift. To this end, we propose a novel, constraint-based method to intuitively specify sequential manipulation tasks and to compute time-optimal robot motions for such a task specification. Our approach follows the ideas of constraint-based task specification by aiming for a minimal and object-centric task description that is largely independent of the underlying robot kinematics. We transform this task description into a non-linear optimization problem. By solving this problem we obtain a (locally) time-optimal robot motion, not just for a single motion, but for an entire manipulation sequence. We demonstrate the capabilities of our approach in a series of experiments involving five distinct robot models, including a highly redundant mobile manipulator.

We give a non-commutative Positivstellensatz for CPn: The (commutative) ⁎-algebra of polynomials on the real algebraic set CPn with the pointwise product can be realized by phase space reduction as the U(1)-invariant polynomials on C1+n, restricted to the real (2n+1)-sphere inside C1+n, and Schmüdgen's Positivstellensatz gives an algebraic description of the real-valued U(1)-invariant polynomials on C1+n that are strictly pointwise positive on the sphere. In analogy to this commutative case, we consider a non-commutative ⁎-algebra of polynomials on C1+n, the Weyl algebra, and give an algebraic description of the real-valued U(1)-invariant polynomials that are positive in certain ⁎-representations on Hilbert spaces of holomorphic sections of line bundles over CPn. It is especially noteworthy that the non-commutative result applies not only to strictly positive, but to all positive (semidefinite) elements. As an application, all ⁎-representations of the quantization of the polynomial ⁎-algebra on CPn, obtained e.g. through phase space reduction or Berezin–Toeplitz quantization, are determined.

We show how combinatorial star products can be used to obtain strict deformation quantizations of polynomial Poisson structures on ℝ^d , generalizing known results for constant and linear Poisson structures to polynomial Poisson structures of arbitrary degree. We give several examples of nonlinear Poisson structures and construct explicit formal star products whose deformation parameter can be evaluated to any real value of ħ , giving strict quantizations on the space of analytic functions on ℝ^d with infinite radius of convergence. We also address further questions such as continuity of the classical limit ħ→ 0 , compatibility with ^* -involutions, and the existence of positive linear functionals. The latter can be used to realize the strict quantizations as ^* -algebras of operators on a pre-Hilbert space which we demonstrate in a concrete example.

For every semisimple coadjoint orbit (O) over cap of a complex connected semisimple Lie group (G) over cap we obtain a family of (G) over cap -invariant products (*) over cap ((h) over bar )on the space of holomorphic functions on (O) over cap. For every semisimple coadjoint orbit O of a real connected semisimple Lie group G, we obtain a family of G-invariant products *((h) over bar ) on a space A(O) of certain analytic functions on O by restriction. A(O), endowed with one of the products *((h) over bar ), is a G-Frechet algebra, and the formal expansion of the products around (h) over bar = 0 determines a formal deformation quantization of O, which is of Wick type if G is compact. Our construction relies on an explicit computation of the canonical element of the Shapovalov pairing between generalized Verma modules and complex analytic results on the extension of holomorphic functions.

Task and motion planning is a relevant yet hard to solve problem in robotic manipulation. Large number of degrees of freedom with multiple manipulators and several objects require specialized algorithms, which can deal with the hybrid planning and optimization problem. An additional challenge is the asynchronous parallelization of single robot actions on interacting manipulators. In this paper we propose a system with a hierarchical planner, which solves the task and motion problem and optimizes for a subsequent parallelization. We use action models based on a constraint formulation; thus, the execution engine can parallelize the sequential plan without synchronization between different tasks. In the experiment, we solve a task and motion problem with difficult geometric constraints and combinatorial complexity. The asynchronously parallel execution of that plan is demonstrated on a real world dual-arm robot.

February 01 2020 IEEE VIS 2016 and 2017 Arts Program Gallery Benedikt Groß, Benedikt Groß Web: https://lab.moovel.com Search for other works by this author on: This Site Google Scholar Raphael Reimann, Raphael Reimann Web: https://lab.moovel.com Search for other works by this author on: This Site Google Scholar Philipp Schmitt, Philipp Schmitt Web: https://lab.moovel.com Search for other works by this author on: This Site Google Scholar Esteban Garcia Bravo, Esteban Garcia Bravo Web: www.carlsongarcia.com Search for other works by this author on: This Site Google Scholar Maxwell Carlson, Maxwell Carlson Web: www.carlsongarcia.com Search for other works by this author on: This Site Google Scholar Aaron Zernack, Aaron Zernack Web: www.carlsongarcia.com Search for other works by this author on: This Site Google Scholar Jorge Garcia, Jorge Garcia Web: www.carlsongarcia.com Search for other works by this author on: This Site Google Scholar Yoon Chung Han, Yoon Chung Han Web: www.yoonchunghan.com Search for other works by this author on: This Site Google Scholar Shankar Tiwari, Shankar Tiwari Web: www.yoonchunghan.com Search for other works by this author on: This Site Google Scholar Till Nagel, Till Nagel Web: https://uclab.fh-potsdam.de/cf Search for other works by this author on: This Site Google Scholar Christopher Pietsch, Christopher Pietsch Web: https://uclab.fh-potsdam.de/cf Search for other works by this author on: This Site Google Scholar Mark J. Stock, Mark J. Stock Web: www.markjstock.org Search for other works by this author on: This Site Google Scholar Weili Shi, Weili Shi Web: www.shi-weili.com Search for other works by this author on: This Site Google Scholar Jessica Parris Westbrook, Jessica Parris Westbrook Web: www.onchanneltwo.com Search for other works by this author on: This Site Google Scholar Adam Trowbridge, Adam Trowbridge Web: www.onchanneltwo.com Search for other works by this author on: This Site Google Scholar Mike Richison, Mike Richison Web: www.mikerichison.com Search for other works by this author on: This Site Google Scholar Mitch Goodwin, Mitch Goodwin Web: www.mitchgoodwin.com Search for other works by this author on: This Site Google Scholar Clement Fay, Clement Fay Web: www.mitchgoodwin.com Search for other works by this author on: This Site Google Scholar Sebastian Lay, Sebastian Lay Web: www.innovis.cpsc.ucalgary.ca Search for other works by this author on: This Site Google Scholar Jo Vermeulen, Jo Vermeulen Web: www.innovis.cpsc.ucalgary.ca Search for other works by this author on: This Site Google Scholar Charles Perin, Charles Perin Web: www.innovis.cpsc.ucalgary.ca Search for other works by this author on: This Site Google Scholar Eric Donovan, Eric Donovan Web: www.innovis.cpsc.ucalgary.ca Search for other works by this author on: This Site Google Scholar Raimund Dachselt, Raimund Dachselt Web: www.innovis.cpsc.ucalgary.ca Search for other works by this author on: This Site Google Scholar Sheelagh Carpendale, Sheelagh Carpendale Web: www.innovis.cpsc.ucalgary.ca Search for other works by this author on: This Site Google Scholar Paul Heinicker, Paul Heinicker Web: www.passim.paulheinicker.com Search for other works by this author on: This Site Google Scholar Dietmar Offenhuber, Dietmar Offenhuber Web: www.offenhuber.net Search for other works by this author on: This Site Google Scholar Duncan Clark, Duncan Clark Web: www.kiln.digital/projects/shipmap Search for other works by this author on: This Site Google Scholar Robin Houston, Robin Houston Web: www.kiln.digital/projects/shipmap Search for other works by this author on: This Site Google Scholar Tristan Smith, Tristan Smith Web: www.kiln.digital/projects/shipmap Search for other works by this author on: This Site Google Scholar Adriene Jenik, Adriene Jenik Web: http://ajenik.faculty.asu.edu Search for other works by this author on: This Site Google Scholar Clarissa Ribeiro, Clarissa Ribeiro Web: www.clarissaribeiro.com Search for other works by this author on: This Site Google Scholar Mick Lorusso, Mick Lorusso Web: www.clarissaribeiro.com Search for other works by this author on: This Site Google Scholar Herbert Rocha, Herbert Rocha Web: www.clarissaribeiro.com Search for other works by this author on: This Site Google Scholar Wonyoung So, Wonyoung So Web: http://wonyoung.so Search for other works by this author on: This Site Google Scholar Mauro Martino, Mauro Martino www.formafluens.io Search for other works by this author on: This Site Google Scholar Hendrik Strobelt, Hendrik Strobelt www.formafluens.io Search for other works by this author on: This Site Google Scholar Owen Cornec, Owen Cornec www.formafluens.io Search for other works by this author on: This Site Google Scholar Scottie Chih-Chieh Huang, Scottie Chih-Chieh Huang Web: www.scottiehuang.com Search for other works by this author on: This Site Google Scholar Yu-Chun Huang, Yu-Chun Huang Web: www.scottiehuang.com Search for other works by this author on: This Site Google Scholar Inhye Lee, Inhye Lee Web: www.inhyelee.com Search for other works by this author on: This Site Google Scholar Hyomin Kim, Hyomin Kim Web: www.inhyelee.com Search for other works by this author on: This Site Google Scholar Pierre Amelot, Pierre Amelot Web: www.vijks.com Search for other works by this author on: This Site Google Scholar John Hwong, John Hwong Web: www.vijks.com Search for other works by this author on: This Site Google Scholar Kate McManus, Kate McManus Web: www.vijks.com Search for other works by this author on: This Site Google Scholar Ryan McGee, Ryan McGee Web: www.lifeorange.com Search for other works by this author on: This Site Google Scholar Mary Bates Neubauer Mary Bates Neubauer Web: www.marybatesneubauer.com Search for other works by this author on: This Site Google Scholar Author and Article Information Benedikt Groß Web: https://lab.moovel.com Raphael Reimann Web: https://lab.moovel.com Philipp Schmitt Web: https://lab.moovel.com Esteban Garcia Bravo Web: www.carlsongarcia.com Maxwell Carlson Web: www.carlsongarcia.com Aaron Zernack Web: www.carlsongarcia.com Jorge Garcia Web: www.carlsongarcia.com Yoon Chung Han Web: www.yoonchunghan.com Shankar Tiwari Web: www.yoonchunghan.com Till Nagel Web: https://uclab.fh-potsdam.de/cf Christopher Pietsch Web: https://uclab.fh-potsdam.de/cf Mark J. Stock Web: www.markjstock.org Weili Shi Web: www.shi-weili.com Jessica Parris Westbrook Web: www.onchanneltwo.com Adam Trowbridge Web: www.onchanneltwo.com Mike Richison Web: www.mikerichison.com Mitch Goodwin Web: www.mitchgoodwin.com Clement Fay Web: www.mitchgoodwin.com Sebastian Lay Web: www.innovis.cpsc.ucalgary.ca Jo Vermeulen Web: www.innovis.cpsc.ucalgary.ca Charles Perin Web: www.innovis.cpsc.ucalgary.ca Eric Donovan Web: www.innovis.cpsc.ucalgary.ca Raimund Dachselt Web: www.innovis.cpsc.ucalgary.ca Sheelagh Carpendale Web: www.innovis.cpsc.ucalgary.ca Paul Heinicker Web: www.passim.paulheinicker.com Dietmar Offenhuber Web: www.offenhuber.net Duncan Clark Web: www.kiln.digital/projects/shipmap Robin Houston Web: www.kiln.digital/projects/shipmap Tristan Smith Web: www.kiln.digital/projects/shipmap Adriene Jenik Web: http://ajenik.faculty.asu.edu Clarissa Ribeiro Web: www.clarissaribeiro.com Mick Lorusso Web: www.clarissaribeiro.com Herbert Rocha Web: www.clarissaribeiro.com Wonyoung So Web: http://wonyoung.so Mauro Martino www.formafluens.io Hendrik Strobelt www.formafluens.io Owen Cornec www.formafluens.io Scottie Chih-Chieh Huang Web: www.scottiehuang.com Yu-Chun Huang Web: www.scottiehuang.com Inhye Lee Web: www.inhyelee.com Hyomin Kim Web: www.inhyelee.com Pierre Amelot Web: www.vijks.com John Hwong Web: www.vijks.com Kate McManus Web: www.vijks.com Ryan McGee Web: www.lifeorange.com Mary Bates Neubauer Web: www.marybatesneubauer.com Online Issn: 1530-9282 Print Issn: 0024-094X ©2020 ISAST2020ISAST Leonardo (2020) 53 (1): 6–24. https://doi.org/10.1162/leon_a_01837 Cite Icon Cite Permissions Share Icon Share Facebook Twitter LinkedIn MailTo Views Icon Views Article contents Figures & tables Video Audio Supplementary Data Peer Review Search Site Citation Benedikt Groß, Raphael Reimann, Philipp Schmitt, Esteban Garcia Bravo, Maxwell Carlson, Aaron Zernack, Jorge Garcia, Yoon Chung Han, Shankar Tiwari, Till Nagel, Christopher Pietsch, Mark J. Stock, Weili Shi, Jessica Parris Westbrook, Adam Trowbridge, Mike Richison, Mitch Goodwin, Clement Fay, Sebastian Lay, Jo Vermeulen, Charles Perin, Eric Donovan, Raimund Dachselt, Sheelagh Carpendale, Paul Heinicker, Dietmar Offenhuber, Duncan Clark, Robin Houston, Tristan Smith, Adriene Jenik, Clarissa Ribeiro, Mick Lorusso, Herbert Rocha, Wonyoung So, Mauro Martino, Hendrik Strobelt, Owen Cornec, Scottie Chih-Chieh Huang, Yu-Chun Huang, Inhye Lee, Hyomin Kim, Pierre Amelot, John Hwong, Kate McManus, Ryan McGee, Mary Bates Neubauer; IEEE VIS 2016 and 2017 Arts Program Gallery. Leonardo 2020; 53 (1): 6–24. doi: https://doi.org/10.1162/leon_a_01837 Download citation file: Ris (Zotero) Reference Manager EasyBib Bookends Mendeley Papers EndNote RefWorks BibTex toolbar search Search Dropdown Menu toolbar search search input Search input auto suggest filter your search All ContentAll JournalsLeonardo Search Advanced Search This content is only available as a PDF. ©2020 ISAST2020ISAST Article PDF first page preview Close Modal You do not currently have access to this content.

In this paper, we present an integrated, model-based system for state estimation and control in dynamic manipulation tasks with partial observability. We track a belief over the system state using a particle filter from which we extract a Gaussian Mixture Model (GMM). This compressed representation of the belief is used to automatically create a discrete set of goal-directed motion controllers. A reinforcement learning agent then switches between these motion controllers in real-time to accomplish the manipulation task. The proposed system closes the loop from joint sensor feedback to high-frequency, acceleration-limited position commands, thus eliminating the need for pre and post-processing. We evaluate our approach with respect to five distinct manipulation tasks from the domains of active localization, grasping under uncertainty, assembly, and non-prehensile object manipulation. Extensive simulations demonstrate that the hierarchical policy actively exploits the uncertainty information encoded in the compressed belief. Finally, we validate the proposed method on a real -world robot.

In this paper, we present an approach to generate robot motions for robust parts assembly. The computation of motions for parts assembly usually requires an exact model of all relevant objects. Generating detailed object models, including friction and dynamics, is often complex and time-consuming, especially in the context of elastic parts. In addition, executing motions on real hardware will usually introduce further uncertainty. For this reason, we propose an approach that is inherently robust against model parameter uncertainties and unknown characteristics of elastic parts. Our planner explicitly takes into account the internal states of articulated objects, as well as uncertain model parameters, by constructing a search tree in the belief-parameter-space. It yields successful assembly motions from coarse object models and thus eliminates the need for detailed parameter tuning. We evaluated our approach with respect to four assembly tasks. Extensive simulations show that our planner significantly increases the success-rate compared to previous approaches. Numerous experiments on a real robot confirm the simulated results.

We obtain a strict quantization of the holomorphic functions on any semisimple coadjoint orbit of a complex semisimple connected Lie group. By restricting this quantization, we also obtain strict star products on a subalgebra of analytic functions for any semisimple coadjoint orbit of a real semisimple connected Lie group. If this Lie group was also compact, the star product is of Wick type. The main tool to construct our quantization is a construction by Alekseev--Lachowska and an explicit formula for the canonical element of the Shapovalov pairing between generalized Verma modules.

When robots perform manipulation tasks, they need to determine their own movement, as well as how to make and break contact with objects in their environment. Reasoning about the motions of robots and objects simultaneously leads to a constrained planning problem in a high-dimensional state-space. Additionally, when environments change dynamically motions must be computed in real-time. To this end, we propose a feedback planner for manipulation. We model manipulation as constrained motion and use this model to automatically derive a set of constraint-based controllers. These controllers are used in a switching-control scheme, where the active controller is chosen by a reinforcement learning agent. Our approach is capable of addressing tasks with second-order dynamics, closed kinematic chains, and time-variant environments. We validated our approach in simulation and on a real, dual-arm robot. Extensive simulation of three distinct robots and tasks show a significant increase in robustness compared to a previous approach.

Abstract In this paper, we discuss continuity properties of the Wick-type star product on the 2-sphere, interpreted as a coadjoint orbit. Star products on coadjoint orbits in general have been constructed by different techniques. We compare the constructions of Alekseev–Lachowska and Karabegov, and we prove that they agree in general. In the case of the 2-sphere, we establish the continuity of the star product, thereby allowing for a completion to a Fréchet algebra.

This paper introduces a Bayesian state estimator for contact-rich manipulation tasks with application in non-prehensile manipulation, industrial assembly or in-hand localization. The core idea of our approach is to explicitly model both the contact dynamics and a torque-based robot controller as part of the underlying system model. Our approach is capable of estimating the state of movable objects for various robot kinematics and geometries of robots and objects. This includes complex scenarios with multiple robots, multiple objects and articulated objects. We have validated our approach in simulation and on a physical robot. The experiments show that multi-modal distributions of six degrees of freedom object poses can be accurately tracked in real-time in a complex manipulation scenario.

In this paper we propose a new model for sequential manipulation tasks that also considers robot dynamics and time-variant environments. From this model we automatically derive constraint-based controllers and use them as steering functions in a kinodynamic manipulation planner. The resulting plan is not a trajectory but a sequence of controllers that react online to disturbances. We validated our approach in simulation and on a real robot. In the experiments our approach plans and executes dual-robot manipulation tasks with online collision avoidance and reactions to estimates of object poses.