Effective Visual Localization (VL) requires a map of the environment that combines compactness for efficient scalability with robustness against visual appearance changes and metric precision. Through low-dimensional image embeddings, Visual Place Recognition (VPR) is able to successfully meet the first two requirements, but its low metric accuracy makes it less suitable than standard VL approaches based on local features or neural representations. This limitation can be overcome by integrating VPR with the accurate local trajectory estimates produced by feed-forward neural 3D geometry (FF3D) models. In this paper, we address sequential appearance-based localization through a topometric framework that iteratively combines probabilistic VPR with FF3D metric pose estimation in controlled image sets. Our approach proposes an automatic offline mapping tool that models the topometric pose-appearance interaction in the different parts of the scene. This map is later employed by an online particle filter that estimates the pose from odometry and belief over places for FF3D inference, successfully incorporating neural metric estimation into probabilistic appearance-based localization. We extensively evaluate the framework on three known benchmarks, demonstrating substantial improvements over existing appearance-based methods. The modularity of our approach allows the descriptor extractor and FF3D model to remain interchangeable, and a focused analysis further shows that sequential belief can mitigate severe failures under perceptual aliasing.

La navegaci´on segura es un aspecto cr´ıtico para los robots sociales que se desplazan en entornos concurridos. Adem´as, estosrobots deben respetar los espacios personales para evitar generar desconfianza y, al mismo tiempo, interactuar de forma efectiva ysocialmente aceptable. Este trabajo presenta un sistema multimodal que combina t´ecnicas de visi´on por computador, planificaci´onde trayectorias socialmente aceptables y estrategias de comunicaci´on proactiva con las personas cercanas al robot. El sistema hasido validado en escenarios controlados, mostrando resultados preliminares prometedores en t´erminos de robustez y fluidez social.

This paper presents a Top-view Transformation Model (TTM) for a vision-based autonomous navigation of an agricultural mobile robot. The TTM transforms images captured by an onboard camera into a virtual Top-view, eliminating perspective distortions such as the vanishing point effect and ensuring uniform pixel distribution. The transformed images are analyzed to ensure an autonomous navigation of the robot between crop rows. The navigation method involves real-time estimation of the robot's position relative to crop rows and the control low is derived from the estimated robot's heading and lateral offset for steering the robot along the crop rows. A simulated scenario has been generated in Gazebo in order to implement the developed approach using the Robot Operating System (ROS), while an evaluation on a real agricultural mobile robot has also been performed. The experimental results demonstrate the feasibility of the TTM approach and its implementation for autonomous navigation, reaching good performance.

Con el objetivo de lograr una interacción humano-robot lo más natural posible, es fundamental que el robot se oriente hacia su interlocutor. Este trabajo presenta un sistema multimodal que usa información visual y de sonido para lograr una orientación precisa incluso en situaciones complejas con múltiples personas, personas fuera del campo de visión del sensor, etc. En concreto, un sistema de micrófonos estéreo es el encargado de detectar el inicio y fin de la interacción, así como de calcular el ángulo de incidencia del sonido para iniciar la orientación del robot. Por su parte, la información visual proveniente de una cámara se usa para localizar la presencia del interlocutor mediante detección facial, asistida por el ángulo de incidencia del sonido. Una vez localizado, el sistema se encarga de orientarse constantemente hacia dicha persona de manera precisa. El trabajo incluye una demostración del comportamiento del sistema en escenarios límite utilizando el robot social Sancho.

El SLAM visual se basa comúnmente en la optimización de un grafo de keyframes, imágenes clave en una secuencia de vídeo, para la construcción de mapas 3D y la localización de la cámara. La creación de este grafo requiere de un proceso front-end eficiente que seleccione keyframes con suficiente solape entre observaciones pero con bajo coste computacional. Este trabajo propone utilizar información de barridos láser 2D, comunes en robots móviles, para determinar el solape entre observaciones de una cámara, incorporando información sobre la estructura de la escena y seleccionando nodos distanciados entre sí cuando el solape cae bajo cierto umbral. Esto evita el costoso proceso de extracción y emparejamiento de características en las imágenes para determinar la covisibilidad entre observaciones. Las pruebas realizadas demuestran que podemos mantener el nivel de solape alrededor de un determinado umbral durante la navegación, insertando keyframes a diferentes frecuencias según la estructura de la escena con un coste computacional significativamente menor que con el uso exclusivo de imágenes.

Representing the scene appearance by a global image descriptor (BoW, NetVLAD, etc.) is a widely adopted choice to address Visual Place Recognition (VPR). The main reasons are that appearance descriptors can be effectively provided with radiometric and perspective invariances as well as they can deal with large environments because of their compactness. However, addressing metric localization with such descriptors (a problem called Appearance-based Localization or AbL) achieves much poorer accuracy than those techniques exploiting the observation of 3D landmarks, which represent the standard for visual localization. In this paper, we propose ALLOM (Appearance-based Localization with Local Observation Models) which addresses AbL by leveraging the topological location of a robot within a map to achieve accurate metric estimations. This topology-assisted metric localization is implemented with a sequential Monte Carlo Bayesian filter that applies a specific observation model for each different place of the environment, thus taking advantage of the local correlation between the pose and the appearance descriptor within each region. ALLOM also benefits from the topological structure of the map to detect eventual robot loss-of-tracking and to effectively cope with its relocalization by applying VPR. Our proposal demonstrates superior metric localization capability compared to different state-of-the-art AbL methods under a wide range of situations.

In this paper, we propose a system that stems from the integration of an autonomous mobile robot with an IoT-based monitoring system to provide monitoring, assistance, and stimulation to older adults living alone in their own houses. The creation of an Internet of Robotics Things (IoRT) based on the interplay between pervasive smart objects and autonomous robotic systems is claimed to enable the creation of innovative services conceived for assisting the final user, especially in elderly care. The synergy between IoT and a Socially Assistive Robot (SAR) was conceived to offer robustness, reconfiguration, heterogeneity, and scalability, by bringing a strong added value to both the current SAR and IoT technologies. First, we propose a method to achieve the synergy and integration between the IoT system and the robot; then, we show how our method increases the performance and effectiveness of both to provide long-term support to the older adults. To do so, we present a case-study, where we focus on the detection of signs of the frailty syndrome, a set of vulnerabilities typically conveyed by a cognitive and physical decline in older people that concur in amplifying the risks of major diseases hindering the capabilities of independent living. Experimental evaluation is performed in both controlled settings and in a long-term real-world pilot study with 9 older adults in their own apartments, where the system was deployed autonomously for, on average, 12 weeks.

Visual Place Recognition (VPR), the task of identifying the place where an image has been taken from, is at the core of important robotic problems as relocalization, loop-closure detection or topological navigation. Even for indoors, the focus of this work, VPR is challenging for a number of reasons, including real-time performance when dealing with large image databases ( $\sim10^{4}$ ) (probably captured by different robots), or the avoidance of Perceptual Aliasing in environments with repetitive structures and scenes. In this letter, we tackle these issues by proposing an off-line mapping technique that abstracts a dense database of georeferenced images without particular order into a Multivariate Gaussian Mixture Model, by creating soft clusters in terms of their similarity in both pose and appearance. This abstract representation is obtained through an Expectation-Maximization algorithm and plays the role of a simplified map. Since querying this map yields a probability of being in a cluster, we exploit this “belief” within a Bayesian filter that regards previous query images and a topological map between clusters to perform more robust VPR. We evaluate our proposal in two different indoor datasets, demonstrating comparable VPR precision to querying the full database while incurring in shorter query times and handling Perceptual Aliasing for sequential navigation.

In an ageing society, the at-home use of Socially Assistive Robots (SARs) could provide remote monitoring of their users’ well-being, together with physical and psychological support. However, private home environments are particularly challenging for SARs, due to their unstructured and dynamic nature which often contributes to robots’ failures. For this reason, even though several prototypes of SARs for elderly care have been developed, their commercialisation and wide-spread at-home use are yet to be effective. In this paper, we analyse how including the end users’ feedback impacts the SARs reliability and acceptance. To do so, we introduce a Monitoring and Logging System (MLS) for remote supervision, which increases the explainability of SAR-based systems deployed in older adults’ apartments, while also allowing the exchange of feedback between caregivers, technicians, and older adults. We then present an extensive field study showing how long-term deployment of autonomous SARs can be accomplished by relying on such a feedback loop to address any potential issue. To this end, we provide the results obtained in a 130-week long study where autonomous SARs were deployed in the apartments of 10 older adults, with the aim of possibly serving and assisting future practitioners, with the knowledge collected from this extensive experimental campaign, to fill the gap that currently exists for the widespread adoption of SARs.

Appearance-based Localization (AL) focuses on estimating the pose of a camera from the information encoded in an image, treated holistically. However, the high-dimensionality of images makes this estimation intractable and some technique of dimensionality Reduction (DR) must be applied. The resulting reduced image representation, though, must keep underlying information about the structure of the scene to be able to infer the camera pose. This work explores the problem of DR in the context of AL, and evaluates four popular methods in two simple cases on a synthetic environment: two linear (PCA and MDS) and two non-linear, also known as Manifold Learning methods (LLE and Isomap). The evaluation is carried out in terms of their capability to generate lower-dimensional embeddings that maintain underlying information that is isometric to the camera poses.

This paper presents an autonomous navigation method for an agricultural mobile robot "AgriEco Robot", with four-wheel-drive and embedded perception sensors. The proposed method allows an accurate guidance between strawberry crop rows while automatically spraying pesticides, as well as detecting the end and switching to the next rows. The main control system was developed using Robot Operating System (ROS) based on a 2D LIDAR sensor. The acquired 2D point clouds data is processed to estimate the robot's heading and lateral offset relative to crop rows. A motion controller is incorporated to ensure the developed autonomous navigation method. Performance in terms of accuracy of the autonomous navigation has been evaluated in real-world conditions within strawberry greenhouses, proving its usefulness for automatic pesticide spraying.

The fusion of visual and inertial measurements is becoming more and more popular in the robotics community since both sources of information complement each other well. However, in order to perform this fusion, the biases of the Inertial Measurement Unit (IMU) as well as the direction of gravity must be initialized first. In case of a monocular camera, the metric scale is also needed. The most popular visual-inertial initialization approaches rely on accurate vision-only motion estimates to build a non-linear optimization problem that solves for these parameters in an iterative way. In this letter, we rely on the previous work in [1] and propose an analytical solution to estimate the accelerometer bias, the direction of gravity and the scale factor in a maximum-a-posteriori framework. This formulation results in a very efficient estimation approach and, due to the non-iterative nature of the solution, avoids the intrinsic issues of previous iterative solutions. We present an extensive validation of the proposed IMU initialization approach and a performance comparison against the state-of-the-art approaches described in [2] and [3] with real data from the publicly available EuRoC dataset. Our approach achieves better accuracy without requiring an initial guess for the scale factor and incorporates a prior for the accelerometer bias in order to avoid observability issues. In terms of computational efficiency, it is as fast as the first work and two times faster than the second. We also provide a C++ open source reference implementation.

A human motion capture system using an RGB-D camera could be a good option to understand the trunk limitations in spondyloarthritis. The aim of this study is to validate a human motion capture system using an RGB-D camera to analyse trunk movement limitations in spondyloarthritis patients. Cross-sectional study was performed where spondyloarthritis patients were diagnosed with a rheumatologist. The RGB-D camera analysed the kinematics of each participant during seven functional tasks based on rheumatologic assessment. The OpenNI2 library collected the depth data, the NiTE2 middleware detected a virtual skeleton and the MRPT library recorded the trunk positions. The gold standard was registered using an inertial measurement unit. The outcome variables were angular displacement, angular velocity and lineal acceleration of the trunk. Criterion validity and the reliability were calculated. Seventeen subjects (54.35 (11.75) years) were measured. The Bending task obtained moderate results in validity (r = 0.55–0.62) and successful results in reliability (ICC = 0.80–0.88) and validity and reliability of angular kinematic results in Chair task were moderate and (r = 0.60–0.74, ICC = 0.61–0.72). The kinematic results in Timed Up and Go test were less consistent. The RGB-D camera was documented to be a reliable tool to assess the movement limitations in spondyloarthritis depending on the functional tasks: Bending task. Chair task needs further research and the TUG analysis was not validated. Graphical abstract Comparation of both systems, required software for camera analysis, outcomes and final results of validity and reliability of each test.

Images of a given environment, coded by a holistic image descriptor, produce a manifold that is articulated by the camera pose in such environment. The correct articulation of such Descriptor Manifold (DM) by the camera poses is the cornerstone for precise Appearance-based Localization (AbL), which implies knowing the correspondent descriptor for any given pose of the camera in the environment. Since such correspondences are only given at sample pairs of the DM (the appearance map), some kind of regression must be applied to predict descriptor values at unmapped locations. This is relevant for AbL because this regression process can be exploited as an observation model for the localization task. This paper analyses the influence of a number of parameters involved in the approximation of the DM from the appearance map, including the sampling density, the method employed to regress values at unvisited poses, and the impact of the image content on the DM structure. We present experimental evaluations of diverse setups and propose an image metric based on the image derivatives, which allows us to build appearance maps in the form of grids of variable density. A preliminary use case is presented as an initial step for future research.

In this paper, we present an algorithm for the detection of line segments directly on the original, distorted images captured by calibrated wide-angle, fisheye and omnidirectional cameras. Distorted line segments are detected as convex polygonal chains of connected straight lines and then validated as the projection of 3D lines. This last validation step is our main contribution, which is formulated in a generic way in order to allow the detection of line segments from calibrated central projection vision systems and without requiring the rectification of the whole image. We evaluate our method with real images from a publicly available dataset and compare it with state-of-the-art alternatives, achieving comparable line detection performance without requiring image rectification. Additionally, we provide an open source reference implementation.

This paper addresses appearance-based robot localization in 2D with a sparse, lightweight map of the environment composed of descriptor–pose image pairs. Based on previous research in the field, we assume that image descriptors are samples of a low-dimensional Descriptor Manifold that is locally articulated by the camera pose. We propose a piecewise approximation of the geometry of such Descriptor Manifold through a tessellation of so-called Patches of Smooth Appearance Change (PSACs), which defines our appearance map. Upon this map, the presented robot localization method applies both a Gaussian Process Particle Filter (GPPF) to perform camera tracking and a Place Recognition (PR) technique for relocalization within the most likely PSACs according to the observed descriptor. A specific Gaussian Process (GP) is trained for each PSAC to regress a Gaussian distribution over the descriptor for any particle pose lying within that PSAC. The evaluation of the observed descriptor in this distribution gives us a likelihood, which is used as the weight for the particle. Besides, we model the impact of appearance variations on image descriptors as a white noise distribution within the GP formulation, ensuring adequate operation under lighting and scene appearance changes with respect to the conditions in which the map was constructed. A series of experiments with both real and synthetic images show that our method outperforms state-of-the-art appearance-based localization methods in terms of robustness and accuracy, with median errors below 0.3 m and 6°.

In domestic robotics, passing through narrow areas becomes critical for safe and effective robot navigation. Due to factors like sensor noise or miscalibration, even if the free space is sufficient for the robot to pass through, it may not see enough clearance to navigate, hence limiting its operational space. An approach to facing this is to insert waypoints strategically placed within the problematic areas in the map, which are considered by the robot planner when generating a trajectory and help to successfully traverse them. This is typically carried out by a human operator either by relying on their experience or by trial-and-error. In this paper, we present an automatic procedure to perform this task that: (i) detects problematic areas in the map and (ii) generates a set of auxiliary navigation waypoints from which more suitable trajectories can be generated by the robot planner. Our proposal, fully compatible with the robotic operating system (ROS), has been successfully applied to robots deployed in different houses within the H2020 MoveCare project. Moreover, we have performed extensive simulations with four state-of-the-art robots operating within real maps. The results reveal significant improvements in the number of successful navigations for the evaluated scenarios, demonstrating its efficacy in realistic situations.

espanolEl reconocimiento de objetos es algo innato en el ser humano. Cuando las personas miramos una fotografia, somos capaces de detectar sin esfuerzo elementos como animales, senales, objetos de interes, etc. En el campo de la vision por computador este proceso se lleva a cabo mediante herramientas de Inteligencia Artificial, con el fin de obtener informacion sobre el contenido de una imagen. Esta tarea, aunque ampliamente investigada, aun sigue siendo un campo de estudio activo debido a los grandes retos que conlleva: la deteccion de objetos en distintas condiciones luminosas, con posibles oclusiones, distintos tamanos y perspectivas, etc. Este articulo describe las tareas a completar en el desarrollo de un sistema de reconocimiento de objetos exitoso, y proporciona al lector una serie de directrices practicas sobre como realizarlas. El trabajo viene acompanado de una serie de scripts Python para experimentar con las diferentes tecnicas descritas, pretendiendo servir de apoyo en tareas docentes o de iniciacion a cualquier entusiasta en la materia. EnglishWhen people look at pictures, we are able to effortlessly detect elements like animals, signals, objects of interest, etc. Object recognition is a computer vision technique that, through tools from Artificial Intelligence, aims to carry out such an action with the goal of retrieving information from the content of an image. This task, although widely investigated, is an active research fied due to the challenges that it has to face: the detection of objects in different lighthing conditions, with possible occlusions, different sizes and perspectives, etc. This paper describes the typical actions to be completed in the development of a successful object recognition system, and provides the reader a number of practical guidelines about how to address them. The work is accompanied by a Python script (available at https: // github. com/ jotaraul/ object_ recognition_ in_ python ) to put them into practice using state-of-the-art techniques (from libraries like Pandas or Scikit-learn ), and aims to be a valuable resource for education, or as initiation to any enthusiastic in the topic.

The detection of object edges in images is a crucial step employed in a vast amount of computer vision applications, for which a series of different algorithms has been developed in the last decades. This paper proposes a new edge detection method based on quantum information, which is achieved in two main steps: (i) an image enhancement stage that employs the quantum superposition law and (ii) an edge detection stage based on the probability of photon arrival to the camera sensor. The proposed method has been tested on synthetic and real images devoted to agriculture applications, where Fram & Deutsh criterion has been adopted to evaluate its performance. The results show that the proposed method gives better results in terms of detection quality and computation time compared to classical edge detection algorithms such as Sobel, Kayyali, Canny and a more recent algorithm based on Shannon entropy.

Background: The RGB-D camera is an alternative to asses kinematics in order to obtain objective measurements of functional limitations. The aim of this study is to analyze the validity, reliability, and responsiveness of the motion capture depth camera in sub-acute and chronic low back pain patients. Methods: Thirty subjects (18–65 years) with non-specific lumbar pain were screened 6 weeks following an episode. RGB-D camera measurements were compared with an inertial measurement unit. Functional tests included climbing stairs, bending, reaching sock, lie-to-sit, sit-to-stand, and timed up-and-go. Subjects performed the maximum number of repetitions during 30 s. Validity was analyzed using Spearman’s correlation, reliability of repetitions was calculated by the intraclass correlation coefficient and the standard error of measurement, and receiver operating characteristic curves were calculated to assess the responsiveness. Results: The kinematic analysis obtained variable results according to the test. The time variable had good values in the validity and reliability of all tests (r = 0.93–1.00, (intraclass correlation coefficient (ICC) = 0.62–0.93). Regarding kinematics, the best results were obtained in bending test, sock test, and sit-to-stand test (r = 0.53–0.80, ICC = 0.64–0.83, area under the curve (AUC) = 0.55–84). Conclusion: Functional tasks, such as bending, sit-to-stand, reaching, and putting on sock, assessed with the RGB-D camera, revealed acceptable validity, reliability, and responsiveness in the assessment of patients with low back pain (LBP). Trial registration: ClinicalTrials.gov NCT03293095 “Functional Task Kinematic in Musculoskeletal Pathology” 26 September 2017