Typical convolutional neural network detectors for acoustic signals rely on large quantities of labeled data, which can be expensive and time-intensive to generate. Unsupervised neural network architectures allow for the automatic detection and classification of acoustic signals, which may be especially useful for classifying underwater signals that a typical human labeler may not be familiar with. Here, we used a convolutional autoencoder to automatically detect and classify fish vocalizations in a coral reef off the coast of Hawaii Island. A database of more than 2 million spectrogram segments were generated with durations of one second and frequency ranges of 30–700 Hz, corresponding to the timescale and spectrum of a significant portion of fish calls. The autoencoder was successful in recreating the features of input spectrograms, and k-means clustering was applied to the latent feature space of each sample to automatically generate classes. Visual inspection of example spectrograms within each class confirms that the autoencoder was successful in classifying fish calls, including differentiating between separate call types.

Sounds from fish and invertebrates in coral reefs can create persistent cacophonies that can be recorded for ecosystem monitoring, including during nighttime hours where visual surveys are typically not feasible. Here we use soundscape measurements in Hawaii to demonstrate that multiple coral reef communities are rapidly responsive to shifts in nighttime ambient light, with sustained changes in biological sound between moonrise and moonset. High frequency pulse train sounds from fish (0.5-1.5 kHz) are found to increase during moonlight hours, while low frequency fish vocalizations (0.1-0.3 kHz) and invertebrate sounds (2-20 kHz) are found to decrease during moonlight hours. These discoveries suggest that the rising and setting of the moon triggers regular shifts in coral reef ecosystem interactions. Future acoustic monitoring of reef health may be improved by comparing soundscapes during moonlight and non-moonlight hours, which may provide early indicators of shifts in the relative abundance of separate reef communities.

Passive acoustic monitoring of biological soundscapes offers a long-term view into ecosystem state. This is particularly well studied for coral reefs and tropical littoral systems with evidence for similar capability in temperate and deep ocean biologically rich ecosystems. Monitoring ecosystem state under both climate change impacts and changing human usage is a critical piece of understanding how climate change and human use impact ecosystems. Passive acoustics allow for wide area coverage of an ecosystem heartbeat, and changes in key bioacoustic metrics in coral reefs indicate shifts from healthy coral dominated systems to more degraded systems with increased macroalgal cover. These shifts are typically associated with increased ocean temperatures and/or increased human use. The primary controls on coral reef soundscapes are time of day and year. Here, broad comparisons between warmer and cooler years at long term coral reef monitoring sites in Hawaii will be discussed, as well as summer versus winter biological soundscapes at temperate sites in New England. Reef soundscapes encompass contributions from a wide variety of marine flora and fauna, some of which can be identified to species level through characteristic calls and tracked with a high degree of fidelity through passive acoustics alone.

The natural swimming ability of fish has inspired the development of stealthy and quiet fish-like robots for a range of applications, from defense to wildlife research, where it is desirable to observe animals in a non-threatening way. A realistic fish-robot will benefit from something resembling proprioception, enabling body and fin position to be sensed for the control of swimming actuators. To demonstrate cyber-proprioception, a stretchy neoprene sensing skin embedded with discrete dielectric elastomer stretch sensors could be used. As the fish-robot body bends, the sensor integrated skin stretches resulting in measurable deformation to the sensors. One of the key challenges however, is stretchy sensors that operate reliably and repeatably in a marine environment. In this study we present a sensor design suitable for underwater use, and compare it with the more commonly used structure. Both sensor designs have been simulated in air and fresh water environments, and have been subjected to strain, sensitivity and cross-talk experiments. We show that the new sensor design renders them insensitive to changes in the dielectric properties of the medium in which they are immersed and to cross-talk noise. As a demonstration we then integrate eight of the sensors, 4 on each side, on a submerged underactuated tensegrity fish-like robot based on the Wahoo (Acanthrocybium solandri), a fast pelagic swimmer. Carangiform movement of the robot’s body has been characterized using cameras and this data has been compared with a virtual model of the robot that uses sensor input for real-time model kinematic data. Extremes of shape change associated with escape and prey capture (C-start and S maneuvers) have also been compared. Four angles along the body defined the shape using camera and sensor data. A root mean square error between model and true camera angles of less than 2.62° was calculated for realistic cangiform motion at 0.16 Hz tail frequency. For the extremes of shape, the sensor-based predictions had an R2 -Score of 0.98 for C and 0.99 for S. When the algorithm was tested on new shapes, the R2 score dropped to 0.81 and 0.79 respectively suggesting some model overfitting. The results support the efficacy of the use of a sensing skin for a soft fish-swimming robot.

A series of sea tests and long term passive acoustics monitoring studies have been conducted in Bermuda from 2020–2023. Observations of biological choruses and individual fish call events from sensors in deep water off the flank of the Bermuda Atoll, and alongside shallow sub-tropical reefs, will be discussed. We find striking similarities between sub-tropical Atlantic reefs in Bermuda with documented trends in tropical coral reef soundscapes studies in Hawaii and the tropical Pacific. This observation suggests that evening chorusing and patterns in reef sound and fish calls are potentially pervasive across large portions of the world’s oceans. Deep water calls that appear to be associated with large fish such as groupers were observed on the atoll flank. Evening chorusing at change of light is commonly observed on tropical reefs and also persistent on Bermuda’s sub-tropical reefs. Evening biological chorusing is also evident at deep water sights, indicating that this may not be a strictly littoral biological soundscape feature.

Climate change impacts and mitigation strategies will define our interaction with the oceans this century. Marine carbon sequestration could facilitate the enormous scaling necessary for gigaton-level carbon dioxide (CO2) removal: at least 10 GT/y by 2050 and 20 GT/y by 2100, required just to limit anthropogenic warming to below 2°C. Many proposed marine CO2 removal techniques involve the distributed capture of carbon, i.e., accelerating the biological carbon pump (e.g., iron ocean fertilization or artificial upwelling) or shifting the dissolution equilibrium of CO2 (e.g., ocean alkalinity enhancement). However, technology that enables rapid, inexpensive, persistent and accurate measurement and validation of drawn-down CO2 at the sequestration time- and regional ocean spatial-scales necessary to quantify carbon capture does not exist today. The accuracy and wholeness of these future techniques will be important for assigning financial value to marine CO2 removal processes in a carbon market, as well as enabling thorough evaluation of environmental impacts and a comprehensive understanding of ocean carbon dynamics. I will discuss ARPA-E’s interest in carbon sensing approaches, including passive and active acoustic techniques, which could rapidly quantify ocean carbon flux at scale and introduce powerful tools to address the challenges of mitigating climate impacts at sea.

Biologically complex coastal environments, such as coral reefs, demonstrate an equally rich ambient soundscape. Bioacoustic features of coastal soundscapes are closely tied with relative ecosystem health, functional groups present, and can be linked with specific behaviors. Biological contributions to ambient soundscapes have distinctive qualities as compared to sound associated with physical processes (i.e. wind and wave noise). While some biological components are readily identifiable, such as marine mammal or fish calls, the background noise associated with hundreds of thousands of biological clicks, snaps, and pops is not as well studied but contains a wealth of information about the ecosystem. A 64-element line array with 4.5 kHz design frequency was deployed for several field experiments off the coast of Kona, Hawaii in 2019 and 2020. Soundscape data from Hawaii were compared with comparable omnidirectional time series from Bermuda (2020) and coastal New England rocky reefs (2020–2021). Similarities in certain spectral features associated with biological sound sources were found between these unique ecosystems. The characteristic coral reef evening chorus, or significant increase in sound levels immediately prior to sunset, was consistent in Hawaii and Bermuda with comparable crepuscular changes in coastal New England.

The superior swimming ability of fish has encouraged the development of fish-like robots. To fully capture fish swimming kinematics a continuum under-actuated robot can be used, and there are many examples of such robots in the literature. But for realistic fish-like swimming in strong currents such robots will benefit from closed-loop feedback. We demonstrate how this can be achieved, underwater, using a stretchy neoprene sensing skin with embedded, discrete dielectric elastomer stretch sensors. The latest prototype skin, with 8 sensors, 4 on each side, is currently being evaluated on an underactuated tensegrity fish-like robot driven by a stepper motor. Carangiform movement of the body has been characterized using cameras and this data has been compared with a virtual model of the robot that uses sensor input for real-time model kinematic data. Four angles along the body defined the shape. A root mean square error between model and true camera angles of less than 3° was calculated for realistic cangiform motion at 0.5 Hz tail frequency.

Deep clustering was applied to unlabeled, automatically detected signals in a coral reef soundscape to distinguish fish pulse calls from segments of whale song. Deep embedded clustering (DEC) learned latent features and formed classification clusters using fixed-length power spectrograms of the signals. Handpicked spectral and temporal features were also extracted and clustered with Gaussian mixture models (GMM) and conventional clustering. DEC, GMM, and conventional clustering were tested on simulated datasets of fish pulse calls (fish) and whale song units (whale) with randomized bandwidth, duration, and SNR. Both GMM and DEC achieved high accuracy and identified clusters with fish, whale, and overlapping fish and whale signals. Conventional clustering methods had low accuracy in scenarios with unequal-sized clusters or overlapping signals. Fish and whale signals recorded near Hawaii in February-March 2020 were clustered with DEC, GMM, and conventional clustering. DEC features demonstrated the highest accuracy of 77.5% on a small, manually labeled dataset for classifying signals into fish and whale clusters.

Detecting acoustic transients by signal-to-noise ratio (SNR) becomes problematic in nonstationary ambient noise environments characteristic of coral reefs. An alternate approach presented here uses signal directionality to automatically detect and localize transient impulsive sounds collected on underwater vector sensors spaced tens of meters apart. The procedure, which does not require precise time synchronization, first constructs time-frequency representations of both the squared acoustic pressure (spectrogram) and dominant directionality of the active intensity (azigram) on each sensor. Within each azigram, sets of time-frequency cells associated with transient energy arriving from a consistent azimuthal sector are identified. Binary image processing techniques then link sets that share similar duration and bandwidth between different sensors, after which the algorithm triangulates the source location. Unlike most passive acoustic detectors, the threshold criterion for this algorithm is bandwidth instead of pressure magnitude. Data collected from shallow coral reef environments demonstrate the algorithm's ability to detect SCUBA bubble plumes and consistent spatial distributions of somniferous fish activity. Analytical estimates and direct evaluations both yield false transient localization rates from 3% to 6% in a coral reef environment. The SNR distribution of localized pulses off Hawaii has a median of 7.7 dB and interquartile range of 7.1 dB.

This paper develops and applies a numerical optimization procedure to compute broadband noise-adaptive weights for delay and sum beamforming that are conditioned to maximize the deflection coefficient at the output of a square law detector for a given set of underwater pressure measurements. The resulting optimal weights mitigate the effects of noise and interferers and maximize signal detection. Comparison of the optimal weights with minimum variance distortionless response weights show that the presented algorithm provides higher attenuation of interferers. We also use the noise-adaptive algorithm to find the optimal sparse array geometry for a given number of sensors and aperture. Comparison of the resulting optimal array with coprime, nested, and semi-coprime arrays shows that the proposed sparse array suppresses interferers more than the other sparse arrays.

An unsupervised process is proposed for clustering and identifying fish calls in an acoustically active coral reef soundscape. First, potentially localizable acoustic events were detected on three directional autonomous seafloor acoustic recorders (DASARs), sampled at 1 kHz, using an automatic directional detector. A maximum likelihood localization algorithm was used to remove unlocalizable events. For each localizable event, standard acoustic metrics were extracted from the spectrogram and timeseries, while user-agnostic latent features were extracted from the spectrograms by an undercomplete convolutional autoencoder (CAE) neural network. Unsupervised clustering methods, including K-means and agglomerative hierarchical, identified distinct acoustic classes from both feature sets. The unsupervised clustering process was used to analyze data collected near a Hawaiian coral reef in February 2020. During a 24-h period, about 1 event per second was detected, with diel variation in the number of detected events. Compared to a hand labeled test set, the unsupervised clustering process with standard acoustic metrics was 91% accurate at identifying fish call trains, which were associated with dusk chorusing, as opposed to single pulse calls or Humpback whale sound. The tradeoffs between physical features and CAE latent features for unsupervised call discovery are discussed.

Robotic vehicles capable of transition from aquatic to terrestrial locomotion face considerable challenges associated with propulsive efficiency and performance in each environment. Here we present a morphing amphibious robotic limb that combines the locomotor adaptations of sea turtles for swimming and tortoises for walking. The limb can transform between the streamlined morphology of a sea turtle flipper and the load-bearing geometry of a tortoise leg using a variable stiffness material coupled to a pneumatic actuator system. Herein, we describe the fabrication and characterization of the morphing limb, and quantitatively show how morphing between hydrodynamic and axial-load bearing states can enhance the locomotive performance of a single design over land and in water.

An unsupervised process is described for clustering automatic detections in an acoustically active coral reef soundscape. First, acoustic metrics were extracted from spectrograms and timeseries of each detection based on observed properties of signal types and classified using unsupervised clustering methods. Then, deep embedded clustering (DEC) was applied to fixed-length power spectrograms of each detection to learn features and clusters. The clustering methods were compared on simulated bioacoustic signals for fish calls and whale song units with randomly varied signal parameters and additive white noise. Overlap and density of the handpicked features led to reduced accuracy for unsupervised clustering methods. DEC clustering identified clusters with fish calls, whale song, and events with simultaneous fish calls and whale song, but accuracy was reduced when the class sizes were imbalanced. Both clustering approaches were applied to acoustic events detected on directional autonomous seafloor acoustic recorder (DASAR) sensors on a Hawaiian coral reef in February-March 2020. Unsupervised clustering of handpicked features did not distinguish fish calls from whale song. DEC had high recall and correctly classified a majority of whale song. Manual labels indicated a class imbalance between fish calls and whale song at a 3-to-1 ratio, likely leading to reduced DEC clustering accuracy.

Coral reefs are complicated and understudied acoustic propagation environments. In addition to geometric spreading, there are propagation losses due to bottom attenuation, volumetric scattering, and boundary scattering from bathymetric variability and sea surface roughness. On a Hawaiian coral reef, a field test was performed in shallow water during which low-level tones were projected from an underwater speaker and received by a single hydrophone at various ranges up to 500 m from the source. Acoustic data gathered during the field test were analyzed to characterize the sound propagation environment. Conventional geometric spreading assumptions were challenged for the sloping bathymetry characteristic of the Hawaiian coral reef environment. Geoacoustic parameters of the coral reef environment were extracted from transmission loss measurements using a nonlinear least-squares inversion. Losses were estimated for frequencies of 500 Hz, 2 kHz, 5 kHz, 10 kHz, and 15 kHz.

Biological sources contribute significantly to coral reef ambient noise environments, yet the ecosystem-level mechanisms of temporal and spatial variation in the reef soundscape are not well understood. In this study, subtle shifts in reef ambient noise are examined using unsupervised machine learning on hydrophone array data. Unsupervised learning does not require data labels, but uses nonlinear inference to find explanatory features within the data. A hydrophone array was used to generate spatially filtered time series inputs for machine learning. Video cameras were collocated and time-synced with the hydrophone array to provide nominal ground-truth. We discuss the tradeoff parameters of the unsupervised learning methods. Changes in the dominant data features during the experiment are compared to the video recordings and researcher observations.

Submersible robotics have improved in efficiency and versatility by incorporating features found in aquatic life, ranging from thunniform kinematics to shark skin textures. To fully realize these benefits, sensor systems must be incorporated to aid in object detection and navigation through complex flows. Again, inspiration can be taken from biology, drawing on the lateral line sensor systems and neuromast structures found on fish. To maintain a truly soft-bodied robot, a man-made flow sensor must be developed that is entirely complaint, introducing no rigidity to the artificial "skin." We present a capacitive cupula inspired by superficial neuromasts. Fabricated via lost wax methods and vacuum injection, our 5 mm tall device exhibits a sensitivity of 0.5 pF/mm (capacitance versus tip deflection) and consists of room temperature liquid metal plates embedded in a soft silicone body. In contrast to existing capacitive examples, our sensor incorporates the transducers into the cupula itself rather than at its base. We present a kinematic theory and energy-based approach to approximate capacitance versus flow, resulting in equations that are verified with a combination of experiments and COMSOL simulations.

The impulsive sounds produced by tropical fish are a prominent component of coral reef acoustic environments off Hawaii. The resultant ambient noise field is highly nonstationary, making it difficult to equalize the noise background when implementing standard intensity-based detectors on conventional hydrophones. Here we demonstrate how DIFAR sensors can be used to enhance the contrast between transient fish signals and background ambient noise, permitting simultaneous detection and triangulation of individual pulses. This approach assigns an azimuth to each time-frequency component of a conventional spectrogram, by computing the arctangent of the active intensity measured on two orthogonal axes. The resulting “azigram” can be processed using standard image processing methods to isolate connected regions that share the same azimuth, and to match similar regions on azigrams from nearby DIFAR sensors. The cross-matched bearings can then be used to triangulate the source. The technique is being used to study “hotspots” of fish activity on coral reef pinnacles.

This paper presents a high data-rate underwater acoustic communication system that does not rely on large, powerful, and computationally complex modems. Our small size, weight, and power system is a reconfigurable acoustic modem platform (RAMP) that exploits transducer directivity to reduce the effects of multipath and mutual interference, thus eliminating the need for a traditional channel equalizer. Experimental results are shown to demonstrate the feasibility of RAMP in a shallow-water ocean environment to deliver real-time data transfer.

Ambient noise in very shallow water is often dominated by sound from biological sources. In addition to their economic and aesthetic value, coral reef environments are of interest acoustically due to their persistently loud biological ambient noise levels, relatively high biomass, diversity of soniferous organisms, and their relatively unique spatial nature in which the receiver is often surrounded by large numbers of transient sources, each emitting relevant information. Stemming from work in the Buckingham lab, our investigation of these sounds and their connection with biological processes continues through both directional and point receivers. This talk will discuss how information contained in the transient aspects of these sound fields can reveal the nature of the sources and physical mechanisms behind the generation of their sounds, spatial distribution of benthic communities, dynamic interactions between organisms and changes in the environment arising from human impacts such as the removal of herbivorous fishes.