We present a platform to foster research in active scene understanding, consisting of high-fidelity simulated environments and a simple yet powerful API that controls a mobile robot in simulation and reality. In contrast to static, pre-recorded datasets that focus on the perception aspect of scene understanding, agency is a top priority in our work. We provide three levels of robot agency, allowing users to control a robot at varying levels of difficulty and realism. While the most basic level provides pre-defined trajectories and ground-truth localisation, the more realistic levels allow us to evaluate integrated behaviours comprising perception, navigation, exploration and SLAM. In contrast to existing simulation environments, we focus on robust scene understanding research using our environment interface (BenchBot) that provides a simple API for seamless transition between the simulated environments and real robotic platforms. We believe this scaffolded design is an effective approach to bridge the gap between classical static datasets without any agency and the unique challenges of robotic evaluation in reality. Our BenchBot Environments for Active Robotics (BEAR) consist of 25 indoor environments under day and night lighting conditions, a total of 1443 objects to be identified and mapped, and ground-truth 3D bounding boxes for use in evaluation. BEAR website: https://qcr.github.io/dataset/benchbot-bear-data/ .

Recent Semantic SLAM methods combine classical geometry-based estimation with deep learning-based object detection or semantic segmentation. In this paper we evaluate the quality of semantic maps generated by state-of-the-art class-and instance-aware dense semantic SLAM algorithms whose codes are publicly available and explore the impacts both semantic segmentation and pose estimation have on the quality of semantic maps. We obtain these results by providing algorithms with ground-truth pose and/or semantic segmentation data available from simulated environments. We establish that semantic segmentation is the largest source of error through our experiments, dropping mAP and OMQ performance by up to 74.3% and 71.3% respectively.

This paper presents a complete topological navigation system for a resource-constrained mobile robot like Pepper, based on image memory and the teach-and-repeat paradigm. Image memory is constructed from a set of reference images that are acquired during a prior mapping phase and arranged topologically. A* search is used to find the optimal path between the current location and the destination. The images from the robot’s RGB camera are used to localize within the topological graph, and an Image-Based Visual Servoing (IBVS) control scheme drives the robot to the next node in the graph. Depth images update a local egocentric occupancy grid, and another IBVS controller navigates local free-space. The output of the two IBVS controllers is fused to form the final control command for the robot. We demonstrate real-time navigation for the Pepper robot in an indoor open-plan office environment without the need for accurate mapping and localization. Our core navigation module can run completely onboard the robot (which has quite limited computing capabilities) at 5 Hz without requiring any external computing resources. We have successfully performed navigation trials over 15 days, visiting more than 50 destinations and traveling more than 1200m with a success rate of over 80%. We discuss remaining challenges and openly share our software.

Being able to explore an environment and understand the location and type of all objects therein is important for indoor robotic platforms that must interact closely with humans. However, it is difficult to evaluate progress in this area due to a lack of standardized testing which is limited due to the need for active robot agency and perfect object ground-truth. To help provide a standard for testing scene understanding systems, we present a new robot vision scene understanding challenge using simulation to enable repeatable experiments with active robot agency. We provide two challenging task types, three difficulty levels, five simulated environments and a new evaluation measure for evaluating 3D cuboid object maps. Our aim is to drive state-of-the-art research in scene understanding through enabling evaluation and comparison of active robotic vision systems.

We introduce BenchBot, a novel software suite for benchmarking the performance of robotics research across both photorealistic 3D simulations and real robot platforms. BenchBot provides a simple interface to the sensorimotor capabilities of a robot when solving robotics research problems; an interface that is consistent regardless of whether the target platform is simulated or a real robot. In this paper we outline the BenchBot system architecture, and explore the parallels between its user-centric design and an ideal research development process devoid of tangential robot engineering challenges. The paper describes the research benefits of using the BenchBot system, including: enhanced capacity to focus solely on research problems, direct quantitative feedback to inform research development, tools for deriving comprehensive performance characteristics, and submission formats which promote sharability and repeatability of research outcomes. BenchBot is publicly available (http://benchbot.org), and we encourage its use in the research community for comprehensively evaluating the simulated and real world performance of novel robotic algorithms.

A study was conducted on, “Existing crop productivity of vegetables growing areas of Neelkantha Municipality of Dhading District, Nepal” for estimating productivity status of vegetable growing areas of Neelkantha municipality of Dhading district. Ward number 3 and 12 were purposively selected, with two land category; upland and lowland. Total 30 households were questioned (15 from ward number 12 and 15 from ward number 3) and 30 soil samples were collected from respective farmers using soil sampling auger at the depth of 0-20 cm which was subject to lab test. Based on the laboratory analysis ward number 12 soil pH was found to be neutral (7.0), high in organic matter (4.42%), medium in total nitrogen (0.22%), high in potassium (333.38 kg ha-1)and high in phosphorus (225 kg ha-1). Similarly, ward number 3 soil pH was alkaline (7.8), high in organic matter (5.47%), high in total nitrogen (0.27%), high in potassium (347.71 kg ha-1) and high in phosphorus (118.7 kg ha-1). Altogether, 30 household soil sample was analyzed. Out of these 17% was slightly acidic, 33% was found to be neutral and 50% was alkaline. SOM of 10% of households was very high, 10% was high, 57% was medium, 10% was low and 13% was very low. Total nitrogen percentage of 13% of households was found to be very high, 27% was found to be high, 43% was medium, 3% was low and 14% was found to be very low. Available potassium of 17% of households was very high, 30% was high and 40% was medium, 13% was low. Available phosphorus of 63% of households was very high, 23% was high, 7% was medium and 7% was low. About 60% of farmers reported increasing trend of productivity. Likewise, 20% farmers reported decreasing trend of productivity and remaining 20% confirmed stagnant crop productivity trend. Commercially vegetable cultivated total areas were 8 hectares. Estimated total vegetables production of previous year was 100 tons worth 45 lakhs with an average cost of Rs. 45/ kg. Thus, the crop productivity of 30 households surveyed is 12.5 tons/ hectare. Majority of farmers claimed that the increase in production was due to the application of chemical fertilizers whereas farmers reported decreasing trend of vegetable productivity due to imbalance application of input like chemical fertilizers. Farmers reported the deterioration of soil fertility due to application of chemical fertilizer in an excessive amount. Thus, farmers were suggested to apply recommended dose of fertilizers for the enhancement of vegetable productivity. It was also suggested to the vegetable grower farmers to carry soil analysis periodically and continue addition of organic matters in the field.

This paper presents image-based navigation from an image memory using a combination of line segments and feature points. The environment is represented by a set of key images, which are acquired during a prior mapping phase that defines the path to be followed during the navigation. The switching of key images is done exploiting the common line segments and feature points between the current acquired image and the nearby key images. Based on the key images and the current image, a control law is derived for computing the rotational velocity of a mobile robot during its visual navigation. Using our approach, real-time navigation has been performed in real indoor environment with a Pioneer 3-DX equipped with an on-board perspective camera and the humanoid robot Pepper without the need of accurate mapping and localization nor of 3D reconstruction. We also show that the combination of points and lines increases the number of features that helps in robust and successful navigation especially in those regions where few points or lines can be detected and tracked/matched.

This letter presents a method for image-based navigation from an image memory using line segments as landmarks. The entire navigation process is based on 2-D image information without using any 3-D information at all. The environment is represented by a set of reference images with overlapping landmarks, which are acquired during a prior learning phase. These reference images define the path to follow during the navigation. The switching of reference images is done exploiting the line segment matching between the current acquired image and nearby reference images. Three view matching result is used to compute the rotational velocity of a mobile robot during its navigation by visual servoing. Real-time navigation has been validated inside a corridor and inside a room with a Pioneer 3-DX equipped with an on-board camera. The obtained results confirm the viability of our approach, and verify that accurate mapping and localization are not necessary for a useful indoor navigation as well as that line segments are better features in the structured indoor environment.

This paper presents a complete framework for image-based navigation from an image memory that exploits mutual information and does not need any feature extraction, matching or any 3D information. The navigation path is represented by a set of automatically selected key images obtained during a prior learning phase. The shared information (entropy) between the current acquired image and nearby key images is exploited to switch key images during navigation. Based on the key images and the current image, the control law proposed by [1] is used to compute the rotational velocity of a mobile robot during its qualitative visual navigation. Using our approach, real-time navigation has been performed inside a corridor and inside a room with a Pioneer 3-DX equipped with an on-board perspective camera without the need of accurate mapping and localization.

Computer-aided diagnosis is developed for assessment of allergic rhinitis/rhinoconjunctivitis measuring the relative redness of sclera under application of allergen solution. The patient’s eye images are taken from commercial digital camera. The iris is robustly localized using a gradient-based Hough circle transform. From the center of the pupil, the region of interest within the sclera is extracted using geometric anatomybased a-priori information. The red color pixels are extracted thresholding in the hue, saturation and value color space. Then, redness is measured by taking mean of saturation projected into zero hue. Evaluation is performed with 92 images taken from 13 subjects, 8 responders and 5 non-responders, which were classified according to an experienced otorhinolaryngologist. Provocation is performed with 100, 1,000 and 10,000 AU/ml allergic solution and normalized to control images without provocation. The evaluation yields redness of 1.14, 1.30, 1.60 and 1.04, 1.12, 1.11 for responders and non-responders, respectively. This indicates that our method is suitable as reliable endpoint in controlled clinical trials.

Computer-aided diagnosis is developed for assessment of allergic rhinitis/rhinoconjunctivitis measuring the relative redness of sclera under application of allergen solution. Images of the patient's eye are taken using a commercial digital camera. The iris is robustly localized using a gradient-based Hough circle transform. From the center of the pupil, the region of interest within the sclera is extracted using geometric anatomy-based a-priori information. The red color pixels are extracted thresholding in the hue, saturation and value color space. Then, redness is measured by taking mean of saturation projected into zero hue. Evaluation is performed with 98 images taken from 14 subjects, 8 responders and 6 non-responders, which were classified according to an experienced otorhinolaryngologist. Provocation is performed with 100, 1,000 and 10,000 AU/ml allergic solution and normalized to control images without provocation. The evaluation yields relative redness of 1.01, 1.05, 1.30 and 0.95, 1.00, 0.96 for responders and non-responders, respectively. Variations in redness measurements were analyzed according to alteration of parameters of the image processing chain proving stability and robustness of our approach. The results indicate that the method improves visual inspection and may be suitable as reliable surrogate endpoint in controlled clinical trials.