Naturallanguage interactionwith synthesis based controlfor simulated free-flying robots

For human-robot teams to effectively collaborate when communication is infrequent or intermittent, robots must reliably and independently perform tasks. Work in grounded language communication has enabled robots to interpret and execute instructions provided natural language, leading to more efficient interactions with human collaborators. To improve the reliability of the task execution, some modern approaches leverage Linear Temporal Logic in conjunction with language grounding models to synthesize correct-by-construction reactive controllers. Synthesis failure may indicate either genuine infeasibility of the task or gaps in the task relevant world knowledge; in the latter case, resolution can come via the human sharing their world knowledge. We demonstrate the ability to inject declarative knowledge into a system that uses grounded language to generate specifications for reactive controllers via experiments conducted with the Astrobee platform a simulated space station.