Camera-based ecosystem monitoring can provide data which is rich with insights into species abundance and behavior, but this information is commonly locked behind long analysis times requiring trained experts to identify taxa and habitat conditions. Methods using deep learning offer the ability to process these data more efficiently than relying entirely on human annotators, but commonly require large training datasets to adapt architectures to ecological tasks. In this study, we compare two potential annotation strategies for building small-medium size datasets (<1500 annotations per class) for underwater faunal monitoring with computer vision. We also explore the effects of architecture choice between You Only Look Once (YOLO) versions 8, 9, 10 and 11. When comparing a specialist annotation strategy to a general one, we found that the former offers comparable performance when sufficient annotations are used in training (2.85% difference in F1 scores in validation). When the number of examples of individual classes is limited, our results showed general detection models having better class-wise performance (mean 21.35% improvement in F1 score in validation). For model selection, we did not find significant differences in performance between YOLO versions tested on our dataset, but propose YOLOv9 as a frontrunner based on a weighted ranking incorporating accuracy and various training and efficiency metrics. The study suggests both types of object detectors have their place in ecosystem monitoring, with specialist one favoring projects with sufficient annotations for a narrow group of interest, and general ones introducing more versatility across classes and annotation levels.

There is no doubt that over the last decade, techniques from the field of machine learning have revolutionized how we process and interpret data, especially images and text. For underwater observations acoustics is a primary source of information, and naturally, deep learning methods have been applied to echograms and other acoustics data, but so far with rather modest results. Here, we argue that due to intrinsic properties of acoustic data, substantial advances will likely require research into deep learning methods beyond mere recycling of models and techniques from image processing. Currently, the potential for breakthroughs in method development is hindered by the lack of standard data formats and organization, and even more by the lack of readily available, high quality data sets with established performance goals. To advance the field, these shortcomings should be remedied

This study contributes to advancing the field of automatic fish event recognition in natural underwater videos, addressing the current gap in studying fish interaction and competition, including predator-prey relationships and mating behaviors. We used the corkwing wrasse (Symphodus melops) as a model, a marine species of commercial importance that reproduces in sea-weed nests built and cared for by a single male. These nests attract a wide range of visitors and are the focal point for behavior such as spawning, chasing, and maintenance. We propose a deep learning methodology to analyze the movement trajectories of the nesting male and classify the associated events observed in their natural habitat. Our approach leverages unsupervised pre-training based on diffusion models, leading to improved feature learning. Additionally, we introduce a dataset comprising 16,937 trajectories across 12 event classes, making it the largest in terms of event class diversity. Our results demonstrate the superior performance of our method compared to several deep architectures. The code for the proposed method and the trajectories can be found at https://github.com/NoeCanovi/Fish_Behaviors_Generative_Models.

Acoustic surveys play a pivotal role in fisheries management. During the surveys, acoustic signals are sent into the water and the strength of the reflection, so-called backscatter, is recorded. The collected data are typically annotated manually, a process that is both labor-intensive and time-consuming, to support acoustic target classification (ATC). The primary objective of this study is to develop an annotation-free deep learning model that extracts acoustic features and improves the representation of acoustic data. For this purpose, we adopt a self-supervised method inspired by the Self DIstillation with NO Labels (DINO) model. Extracting useful acoustic features is an intricate task due to the inherent variability and complexity in biological targets, as well as environmental and technical factors influencing sound interactions. The proposed model is trained with three sampling methods: random sampling, which ignores class imbalance present in the acoustic survey data; class-balanced sampling, which ensures equal representation of known categories; and intensity-based sampling, which selects data to capture backscatter variations. The quality of extracted features is then evaluated and compared. We show that the extracted features lead to improvement, in comparison to using the untreated data, in the discriminative power of several machine learning methods (k-nearest neighbor (kNN), linear regression, multinomial logistic regression) for ATC. The improvement was measured through higher accuracy in kNN (77.55% vs. 71.93%), Macro AUC in logistic regression (0.92 vs. 0.80), and R2 in linear regression (0.69 vs. 0.45) when comparing extracted features to the untreated data. Our findings highlight the advantage of applying emerging self-supervised techniques in fisheries acoustics. This study thus contributes to the ongoing efforts to improve the efficiency of acoustic surveys in fisheries management.

Otoliths are a central information source for fish ecology and stock management, conveying important data about age and other life history for individual fish. Traditionally, interpretation of otoliths has required skilled expert readers, but recently deep learning classification and regression models have been trained to extract fish age from images of otoliths from a variety of species. Despite high accuracy in many cases, the adoption of such models in fisheries management has been slow. One reason may be that the underlying mechanisms the model uses to derive its results from the data are opaque, and this lack of legibility makes it challenging to build sufficient trust in the results. Here, we implement a deep learning model that instead of age predicts the location of annotation marks for each of the annuli. This allows an expert to evaluate the model's performance in detail. The quality of the annotations was judged by a panel of four expert otolith readers in a double-blinded randomized survey. Using a scale from 1 to 5, the generated marks received an average quality score of 4.22, whereas expert annotations received an average score of 4.33. By counting the marks to determine fish age, we obtained an agreement between expert and model annotations of 64% on our test set, which running the model stochastically increased to 69%. Stochastic sampling yields further benefits, including an explicit measure of the model's uncertainty, the post hoc likelihood of the different age classes for each otolith, and a set of alternative annotation sequences that highlight the structure of the annuli.

Fish age estimation plays a crucial role in stock management and provides valuable information for biological studies. Fish age is typically estimated by experts that manually count annual increments in otoliths. This process is prone to age reader bias, which makes comparisons between readers and labs challenging, and requires considerable time and resources. In this study, we developed a machine learning framework for fish age prediction using 5150 images of otoliths from Barents Sea Atlantic cod (Gadus morhua) collected between 2012 and 2018. In contrast to previous studies that utilise models trained on otolith sections, we used images of broken otoliths that require no processing prior to imaging, and hence, could potentially facilitate at-sea age estimation. We trained convolutional neural networks (CNNs) based on two modern architectures (EfficientNetV1 and EfficientNetV2), which vary in model size (number of model parameters), and compared performance. Model average accuracy was 72.7% and mean-squared-error was 0.284 when compared with the human-read ages. The models' accuracy for one- and two-year-old individuals was over 90% and no systematic bias in the age predictions across age groups was detected. The best models were EfficientNet B4 and EfficientNet B6 using images taken with low exposure times. A maximum accuracy of 78.6% was achieved using an ensemble consisting of six models. Model predictions were also strongly correlated, limiting the utility of building large ensembles Model performance was compared to the results of an internal workshop where 100 independent images of broken otoliths were aged by a group of experts. Variations in percentage agreement between age classes showed a similar pattern (decreasing with age) in both CNN-based predictions and age estimates made by the expert group. While CNN-based percentage agreement was often lower than the expert estimates, it remained within or close to the range of percentage agreement observed across all readers. Our results demonstrate the potential of deep learning techniques for extracting age estimates from otolith images. When developing frameworks for age estimation using machine learning, we recommend EfficientNet B4 models are used as they are quicker to train than larger models and perform well. Ensemble approaches are also recommended if sufficient computational resources are available, as they can provide increased accuracy and lower variance of the predictions.

Previous studies on robustness have argued that there is a tradeoff between accuracy and adversarial accuracy. The tradeoff can be inevitable even when we neglect generalization. We argue that the tradeoff is inherent to the commonly used definition of adversarial accuracy, which uses an adversary that can construct adversarial points constrained by $\epsilon$-balls around data points. As $\epsilon$ gets large, the adversary may use real data points from other classes as adversarial examples. We propose a Voronoi-epsilon adversary which is constrained both by Voronoi cells and by $\epsilon$-balls. This adversary balances two notions of perturbation. As a result, adversarial accuracy based on this adversary avoids a tradeoff between accuracy and adversarial accuracy on training data even when $\epsilon$ is large. Finally, we show that a nearest neighbor classifier is the maximally robust classifier against the proposed adversary on the training data.

In both terrestrial and marine ecology, physical tagging is a frequently used method to study population dynamics and behavior. However, such tagging techniques are increasingly being replaced by individual re-identification using image analysis. This paper introduces a contrastive learning-based model for identifying individuals. The model uses the first parts of the Inception v3 network, supported by a projection head, and we use contrastive learning to find similar or dissimilar image pairs from a collection of uniform photographs. We apply this technique for corkwing wrasse, Symphodus melops, an ecologically and commercially important fish species. Photos are taken during repeated catches of the same individuals from a wild population, where the intervals between individual sightings might range from a few days to several years. Our model achieves a one-shot accuracy of 0.35, a 5-shot accuracy of 0.56, and a 100-shot accuracy of 0.88, on our dataset.

Most classification models treat all misclassifications equally. However, different classes may be related, and these hierarchical relationships must be considered in some classification problems. These problems can be addressed by using hierarchical information during training. Unfortunately, this information is not available for all datasets. Many classification-based metric learning methods use class representatives in embedding space to represent different classes. The relationships among the learned class representatives can then be used to estimate class hierarchical structures. If we have a predefined class hierarchy, the learned class representatives can be assessed to determine whether the metric learning model learned semantic distances that match our prior knowledge. In this work, we train a softmax classifier and three metric learning models with several training options on benchmark and real-world datasets. In addition to the standard classification accuracy, we evaluate the hierarchical inference performance by inspecting learned class representatives and the hierarchy-informed performance, i.e., the classification performance, and the metric learning performance by considering predefined hierarchical structures. Furthermore, we investigate how the considered measures are affected by various models and training options. When our proposed ProxyDR model is trained without using predefined hierarchical structures, the hierarchical inference performance is significantly better than that of the popular NormFace model. Additionally, our model enhances some hierarchy-informed performance measures under the same training options. We also found that convolutional neural networks (CNNs) with random weights correspond to the predefined hierarchies better than random chance.

Machine learning covers a large set of algorithms that can be trained to identify patterns in data. Thanks to the increase in the amount of data and computing power available, it has become pervasive across scientific disciplines. We first highlight why machine learning is needed in marine ecology. Then we provide a quick primer on machine learning techniques and vocabulary. We built a database of & SIM;1000 publications that implement such techniques to analyse marine ecology data. For various data types (images, optical spectra, acoustics, omics, geolocations, biogeochemical profiles, and satellite imagery), we present a historical perspective on applications that proved influential, can serve as templates for new work, or represent the diversity of approaches. Then, we illustrate how machine learning can be used to better understand ecological systems, by combining various sources of marine data. Through this coverage of the literature, we demonstrate an increase in the proportion of marine ecology studies that use machine learning, the pervasiveness of images as a data source, the dominance of machine learning for classification-type problems, and a shift towards deep learning for all data types. This overview is meant to guide researchers who wish to apply machine learning methods to their marine datasets.

Every year, marine scientists around the world read thousands of otolith or scale images to determine the age structure of commercial fish stocks. This knowledge is important for fisheries and conservation management. However, the age-reading procedure is time-consuming and costly to perform due to the specialized expertise and labor needed to identify annual growth zones in otoliths. Effective automated systems are needed to increase throughput and reduce cost. DeepOtolith is an open-source artificial intelligence (AI) platform that addresses this issue by providing a web system with a simple interface that automatically estimates fish age by combining otolith images with convolutional neural networks (CNNs), a class of deep neural networks that has been a dominant method in computer vision tasks. Users can upload otolith image data for selective fish species, and the platform returns age estimates. The estimates of multiple images can be exported to conduct conclusions or further age-related research. DeepOtolith currently contains classifiers/regressors for three fish species; however, more species will be included as related work on ageing will be tested and published soon. Herein, the architecture and functionality of the platform are presented. Current limitations and future directions are also discussed. Overall, DeepOtolith should be considered as the first step towards building a community of marine ecologists, machine learning experts, and stakeholders that will collaborate to support the conservation of fishery resources.

In metric learning, the goal is to learn an embedding so that data points with the same class are close to each other and data points with different classes are far apart. We propose a distance-ratio-based (DR) formulation for metric learning. Like softmax-based formulation for metric learning, it models $p(y=c|x')$, which is a probability that a query point $x'$ belongs to a class $c$. The DR formulation has two useful properties. First, the corresponding loss is not affected by scale changes of an embedding. Second, it outputs the optimal (maximum or minimum) classification confidence scores on representing points for classes. To demonstrate the effectiveness of our formulation, we conduct few-shot classification experiments using softmax-based and DR formulations on CUB and mini-ImageNet datasets. The results show that DR formulation generally enables faster and more stable metric learning than the softmax-based formulation. As a result, using DR formulation achieves improved or comparable generalization performances.

The genome size of organisms impacts their evolution and biology and is often assumed to be characteristic of a species. Here we present the first published estimates of genome size of the ecologically and economically important ectoparasite, Lepeophtheirus salmonis (Copepoda, Caligidae). Four independent L. salmonis genome assemblies of the North Atlantic subspecies Lepeophtheirus salmonis salmonis , including two chromosome level assemblies, yield assemblies ranging from 665 to 790 Mbps. These genome assemblies are congruent in their findings, and appear very complete with Benchmarking Universal Single-Copy Orthologs analyses finding > 92% of expected genes and transcriptome datasets routinely mapping > 90% of reads. However, two cytometric techniques, flow cytometry and Feulgen image analysis densitometry, yield measurements of 1.3–1.6 Gb in the haploid genome. Interestingly, earlier cytometric measurements reported genome sizes of 939 and 567 Mbps in L. salmonis salmonis samples from Bay of Fundy and Norway, respectively. Available data thus suggest that the genome sizes of salmon lice are variable. Current understanding of eukaryotic genome dynamics suggests that the most likely explanation for such variability involves repetitive DNA, which for L. salmonis makes up ≈ 60% of the genome assemblies.

For several fish species, age and other important biological information is manually inferred from visual scrutinization of scales, and reliable automatic methods are not widely available. Here, we apply Convolutional Neural Networks (CNN) with transfer learning on a novel dataset of 9056 images of Atlantic salmon scales for four different prediction tasks. We predicted fish origin (wild/farmed), spawning history (previous spawner/non-spawner), river age, and sea age. We obtained high prediction accuracy for fish origin (96.70%), spawning history (96.40%), and sea age (86.99%), but lower accuracy for river age (63.20%). Against six human expert readers with an additional dataset of 150 scales, the CNN showed the second-highest percentage agreement for sea age (94.00%, range 87.25±97.30%), but the lowest agreement for river age (66.00%, range 66.00– 84.68%). Estimates of river age by expert readers exhibited higher variance and lower levels of agreement compared to sea age and may indicate why this task is also more difficult for the CNN. Automatic interpretation of scales may provide a cost- and time-efficient method of predicting fish age and life-history traits.

Fish counts and species information can be obtained from images taken within trawls, which enables trawl surveys to operate without extracting fish from their habitat, yields distribution data at fine scale for better interpretation of acoustic results, and can detect fish that are not retained in the catch due to mesh selection. To automate the process of image-based fish detection and identification, we trained a deep learning algorithm (RetinaNet) on images collected from the trawl-mounted Deep Vision camera system. In this study, we focused on the detection of blue whiting, Atlantic herring, Atlantic mackerel, and mesopelagic fishes from images collected in the Norwegian sea. To address the need for large amounts of annotated data to train these models, we used a combination of real and synthetic images, and obtained a mean average precision of 0.845 on a test set of 918 images. Regression models were used to compare predicted fish counts, which were derived from RetinaNet classification of fish in the individual image frames, with catch data collected at 20 trawl stations. We have automatically detected and counted fish from individual images, related these counts to the trawl catches, and discussed how to use this in regular trawl surveys.

Developing high‐performing machine learning algorithms requires large amounts of annotated data. Manual annotation of data is labour‐intensive, and the cost and effort needed are an important obstacle to the development and deployment of automated analysis. In a previous work, we have shown that deep learning classifiers can successfully be trained on synthetic images and annotations. Here, we provide a curated set of fish image data and backgrounds, the necessary software tools to generate synthetic images and annotations, and annotated real datasets to test classifier performance. The dataset is constructed from images collected using the Deep Vision system during two surveys from 2017 and 2018 that targeted economically important pelagic species in the Northeast Atlantic Ocean. We annotated a total of 1,879 images, randomly selected across trawl stations from both surveys, comprising 482 images of blue whiting, 456 images of Atlantic herring, 341 images of Atlantic mackerel, 335 images of mesopelagic fishes and 265 images containing a mixture of the four categories.

Copepods encompass numerous ecological roles including parasites, detrivores and phytoplankton grazers. Nonetheless, copepod genome assemblies remain scarce. Lepeophtheirus salmonis is an economically and ecologically important ectoparasitic copepod found on salmonid fish. We present the 695.4 Mbp L. salmonis genome assembly containing ≈60% repetitive regions and 13,081 annotated protein-coding genes. The genome comprises 14 autosomes and a ZZ-ZW sex chromosome system. Assembly assessment identified 92.4% of the expected arthropod genes. Transcriptomics supported annotation and indicated a marked shift in gene expression after host attachment, including apparent downregulation of genes related to circadian rhythm coinciding with abandoning diurnal migration. The genome shows evolutionary signatures including loss of genes needed for peroxisome biogenesis, presence of numerous FNII domains, and an incomplete heme homeostasis pathway suggesting heme proteins to be obtained from the host. Despite repeated development of resistance against chemical treatments L. salmonis exhibits low numbers of many genes involved in detoxification.

This paper analyzes the problems of adversarial accuracy and adversarial training. We argue that standard adversarial accuracy fails to properly measure the robustness of classifiers. Its definition has a tradeoff with standard accuracy even when we neglect generalization. In order to handle the problems of the standard adversarial accuracy, we introduce a new measure for the robustness of classifiers called genuine adversarial accuracy. It can measure the adversarial robustness of classifiers without trading off accuracy on clean data and accuracy on the adversarially perturbed samples. In addition, it does not favor a model with invariance-based adversarial examples, samples whose predicted classes are unchanged even if the perceptual classes are changed. We prove that a single nearest neighbor (1-NN) classifier is the most robust classifier according to genuine adversarial accuracy for given data and a norm-based distance metric when the class for each data point is unique. Based on this result, we suggest that using poor distance metrics might be one factor for the tradeoff between test accuracy and lp norm-based test adversarial robustness.