A benchmarking perspective of underwater intervention systems

This paper presents recent progress concerning benchmarking issues in the underwater robotics manipulation context. After a very intensive 6-years period of work, under several funding research projects, all of them in the aforementioned area, a strong know-how has been developed. As part of this expertise, a new underwater simulation tool has been implemented. This platform enables the integration, simulation, comparative analysis and experimentation on real data and a detailed characterization of the results, using as input a simple web-based user interface. In fact, our previous experience in related research projects has evidenced the necessity of testing, comparing and evaluating different algorithms in similar conditions. So, the similarity between the virtual scenario, where the benchmarking will be performed, and the real one, will determine the quality of the results. In that sense, a methodology is presented which updates the scenario with real information each time a real trial is performed. Benchmarking, a very active robotic area in nowadays, is the underlying problem to solve. In summary, the main functionalities for benchmarking available in the simulation platform will be highlighted, by using some case studies concerning object tracking under visibility changes, object tracking under variable water current and 3D reconstruction subject to different optical conditions.