Reproducibility and replicability (R&R) are important for research. Many communities are beginning efforts to reward, incentivize, and highlight projects as a motive to adopt R&R practices. This is clearly a good direction - we should all aim to make our research sound, replicable, and reproducible. Yet, this involves a lot of effort to document, debug, and generally make the systems that we build more usable. Interfacing with the Physical world and building custom Things exacerbates these challenges. Therein lies the dilemma: how does the CPS/IoT community reward and incentivize R&R efforts? This paper looks into the question of R&R in CPS/IoT. We survey efforts in other fields spanning computing to healthcare and highlight similarities and differences to CPS/IoT. We then discuss several exemplar CPS/IoT projects related to UCSD's research and highlight the R&R efforts in these projects, the potential ways that they could be improved, and best practices. We finish with recommendations and insights for R&R tailored to the CPS/IoT community.

There is a need for reliable underwater fish monitoring systems that can provide oceanographers and researchers with valuable data about life underwater. Most current methods rely heavily on human observation which is both error prone and costly. FishSense provides a solution that accelerates the use of depth cameras underwater, opening the door to 3D underwater imaging that is fast, accurate, cost effective, and energy efficient. FishSense is a sleek handheld underwater imaging device that captures both depth and color images. This data has been used to calculate the length of fish, which can be used to derive biomass and health. The FishSense platform has been tested through two separate deployments. The first deployment imaged a toy fish of known length and volume within a controlled testing pool. The second deployment was conducted within an 70,000 gallon aquarium tank with multiple species of fish. A Receiver Operating Characteristic (ROC) curve has been computed based on the detector’s performance across all images, and the mean and standard deviation of the length measurements of the detections has been computed.

: Robust object detection in sonar images is an important task for underwater exploration, navigation and mapping. Current methods make assumptions about the shape, highlight or shadow of an object, which may be invalid for some environments or targets. We focus on the area of feature extraction-based detection, which does not rely on information about the shape of the target, towards a robust framework for target detection for a variety of seabed structures and target types. The proposed framework first estimates the seabed type from the spatial distribution of features to determine the set of optimal parameters, and then obtains a set of features which are filtered according to intensity and distribution to yield a detection decision. The proposed method also provides a means to determine the seabed type, and a machine-learning based methodology to choose the feature detectors' parameters to match the evaluated seabed type. We report the performance of a variety of feature detectors for a simulated environment and of one feature detector for real sonar images. Results show the importance of choosing the parameters of the feature extractors based on the current environmental conditions and the proposed method obtains a favourable tradeoff between detection and false alarm rates.

Purpose Digital documentation techniques of tunneling excavations at archaeological sites are becoming more common. These methods, such as photogrammetry and LiDAR (Light Detection and Ranging), are able to create precise three-dimensional models of excavations to complement traditional forms of documentation with millimeter to centimeter accuracy. However, these techniques require either expensive pieces of equipment or a long processing time that can be prohibitive during short field seasons in remote areas. This article aims to determine the effectiveness of various low-cost sensors and real-time algorithms to create digital scans of archaeological excavations. Design/methodology/approach The authors used a class of algorithms called SLAM (Simultaneous Localization and Mapping) along with depth-sensing cameras. While these algorithms have largely improved over recent years, the accuracy of the results still depends on the scanning conditions. The authors developed a prototype of a scanning device and collected 3D data at a Maya archaeological site and refined the instrument in a system of natural caves. This article presents an analysis of the resulting 3D models to determine the effectiveness of the various sensors and algorithms employed. Findings While not as accurate as commercial LiDAR systems, the prototype presented, employing a time-of-flight depth sensor and using a feature-based SLAM algorithm, is a rapid and effective way to document archaeological contexts at a fraction of the cost. Practical implications The proposed system is easy to deploy, provides real-time results and would be particularly useful in salvage operations as well as in high-risk areas where cultural heritage is threatened. Originality/value This article compares many different low-cost scanning solutions for underground excavations, along with presenting a prototype that can be easily replicated for documentation purposes.

Augmented Reality (AR) surgical image guidance overlays preoperative data into the surgeon's view in real time during the procedure. Non-rigid 3D registration is a critical and often challenging step for AR surgical image guidance. Since surgical environments vary greatly and registration must by done quickly and accurately, it is unlikely that one registration technique will work well over different surgical scenarios. Unfortunately, it is currently challenging to evaluate the accuracy and effectiveness of 3D registration techniques on surgical scenes. In this work, we provide a novel method to benchmark quality of non-rigid 3D surface registration. Our method provides a triangular mesh overlay representing the quality of registration and can highlight areas of unacceptably poor registration performance given some specified tolerance. We use the method to evaluate the quality of two existing non-rigid registration approaches on surgical video.

Clock synchronization is important when considering a swarm of underwater vehicles. Unfortunately, accurate clocks are expensive, and acoustic communication is power intensive, preventing large numbers of vehicles from being deployed for long periods of time. We propose a technique where vehicles achieve relative synchronization by leveraging sounds already present in the underwater environment. This provides a synchronization method that does not rely on expensive clocks, communication, or additional infrastructure. We demonstrate that our technique accurately tracks clock offsets between vehicles during an at sea deployment.

In this work we compare the performance of seven popular feature detection algorithms on a synthetic sonar image dataset. The dataset consists of a single mine-like object (MLO) superimposed on three different backgrounds: grass, sand ripple, and sand. We explore the performance of Harris, Shi-Tomasi, SIFT, SURF, STAR, FAST, and ORB on each of these backgrounds, and all the backgrounds at once by training an SVM classifier. Performance is evaluated with ROC curves by comparing the number of correctly identified features belonging to objects (True Positives) and the number of incorrectly identified features belonging to background noise (False Positives).