This paper introduces a comprehensive model training pipeline for Isolated Sign Language Recognition (ISLR) designed to accommodate the distinctive characteristics and constraints of the Sign Language (SL) domain. The constructed pipeline incorporates carefully selected image and video augmentations to tackle the challenges of low data quality and varying sign speeds. Including an additional regression head combined with IoU-balanced classification loss enhances the model's awareness of the gesture and simplifies capturing temporal information. Extensive experiments demonstrate that the developed training pipeline easily adapts to different datasets and architectures. Additionally, the ablation study shows that each proposed component expands the potential to consider ISLR task specifics. The presented strategies improve recognition performance on a broad set of ISLR benchmarks. Moreover, we achieved a state-of-the-art result on the WLASL and Slovo benchmarks with 1.63% and 14.12% improvements compared to the previous best solution, respectively.
In this paper, we aim to improve rendering reflections using environment maps on moving reflective objects. Such scenarios require multiple reflection probes to be positioned at various locations in a scene. During rendering, the closest reflection probe is typically chosen as the environment map of a specific object, resulting in sharp transitions between the rendered reflections when the object moves around the scene. To solve this problem, we developed two convolutional neural networks that dynamically synthesize the best possible environment map at a given point in the scene. The first network generates an environment map from the coordinates of a given point through a decoder architecture. In the second approach, we triangulated the scene and captured environment maps at the triangle vertices - these represent reflection probes. The second network receives at the input three environment maps captured at the vertices of the triangle containing the query point, along with the distances between the query point and the vertices. Through an encoder-decoder architecture, the second network performs smart interpolation of the three environment maps. Both approaches are based on the phenomenon of overfitting, which made it necessary to train each network individually for specific scenes. Both networks are successful at predicting environment maps at arbitrary locations in the scene, even if these locations were not part of the training set. The accuracy of the predictions strongly depends on the complexity of the scene itself.
Modern graphics APIs expose control over the infamously non-coherent GPU caches to application programmers through the mechanisms of pipeline barriers and render passes. A developer is then asked to group together their GPU computations based on memory access patterns such that cache flushes and invalidations are minimized, but render graph systems enable automation of this process. In this paper, we study the problem of finding an optimal execution order for a frame graph to minimize the amount of render pass breaks, which in turn minimizes cache control operations. We formulate and analyze a novel $NP$-complete problem $lang{MLGP}$ and use it to propose an approach to render pass merging that results in 30\% less render pass breaks when compared to previous works.
This paper proposes a natural data augmentation method and an anomaly removal artificial neural network for accurate anomaly detection. Anomaly detection is important because the provision of high-quality products is vital in the manufacturing industry. However, it is difficult to obtain a sufficient number of anomaly samples for the detection, which represents a significant challenge when it comes to achieving accurate anomaly detection by machine learning. General data augmentation methods generate new anomaly images by combining normal images and anomaly images. As an alternative, this paper describes a method that generates new anomaly images by using the Eigenspace. More natural anomaly images are generated than with general data augmentation methods. This paper also proposes an anomaly removal neural network that utilizes this natural data augmentation. The results of an anomaly detection experiment showed that the AUC of 94.7% was achieved for the capsule dataset when using anomaly images generated by the proposed data augmentation for training the anomaly removal neural network. This is 1.3% higher than the state-of-the-art data augmentation method that has been utilized for training the neural network. In the case of the pill dataset, AUC of 99.4% was achieved by proposed method. This is 3.0% higher than the state-of-the-art data augmentation method that has been utilized for training the neural network. The results of a series of experiments demonstrated that anomaly images generated by the proposed data augmentation are effective for training the neural network.
Fish motion is a very important indicator of various health conditions of fish swarms in the fish farming industry. Many researchers have successfully analyzed fish motion information with the help of special sensors or computer vision, but their research results were either limited to few robotic fishes for ground-truth reasons or restricted to 2D space. Therefore, there is still a lack of methods that can accurately estimate the motion of a real fish swarm in 3D space. Here we present our Fish Motion Estimation (FME) algorithm that uses multi-object tracking, monocular depth estimation, and our novel post-processing approach to estimate fish motion in the world coordinate system. Our results show that the estimated fish motion approximates the ground truth very well and the achieved accuracy of 81.0% is sufficient for the use case of fish monitoring in fish farms.
Personality traits are characteristics that can describe a person’s behavior, also reflecting their thoughts and feelings. There are those who support the idea that traits can be strong predictors of leadership, implying emotional stability of the individual. Knowing the importance of the subject, areas such as psychology and neuropsychology have been studying and analyzing personality, aiming to better understand such patterns that guide behavior. A model widely accepted to categorize personality traits is known as Big Five and uses the acronym OCEAN: Openness, Conscientiousness, Extroversion, Agreeableness and Neuroticism. On the other hand, new approaches that emerged from the field of computer vision allow to analyzing personality from visual data, making this new area of research quite attractive for researchers. This work presents an initial study of the use of the Transformer architecture to analyze personality traits, with a specific focus on extroversion, using digital videos of human faces. A literature review was carried out focusing on the application of computational techniques in this issue involving deep learning and Transformers. We also accomplished an experiment analysing Extroversion personality trait, as a starting point for our studies, using the ChaLearn dataset. An AUC (Area under the ROC Curve) value of 71.04% was obtained, with fine adjustment of parameters in the transformer, demonstrating the robustness of the proposed architecture.
Fairing methods, frequently used for smoothing noisy features of surfaces, evolve a surface towards a simpler shape. The process of shaping a simple surface into a more complex object requires using a scalar field defined in the ambient space to drive the surface toward a target shape. Practical implementation of such evolution, referred to as Lagrangian Shrink-Wrapping, on discrete mesh surfaces presents a variety of challenges. Our key innovation lies in the integration of adaptive remeshing and curvature-based feature detection, ensuring mesh quality and proximity to target data all while maintaining the stability of the solution in time. We introduce the Equilateral Triangle Jacobian Condition Number metric for assessing triangle quality and introduce trilinear interpolation for enhanced surface detailing to improve upon existing implementations. Our approach is tested with point cloud meshing, isosurface extraction, and the elimination of internal mesh data, providing significant improvements in efficiency and accuracy. Moreover, we extend the evolution to surfaces with higher genus to shrink-wrap even more complex data.
Strokes concerned more than 795,000 individuals annually in the United States as of 2021. Detecting thrombus (blood clot) is crucial for aiding surgeons in diagnosis, a process heavily reliant on 3D models reconstructed from medical imaging. While these models are very dense with information (many vertices, edges, faces in the mesh, and noise), extracting the critical data is essential to produce an accurate analysis to support the work of practitioners. Our research, conducted in collaboration with a consortium of surgeons, leverages generalized maps (g-maps) to compute quality criteria on the cerebral vascular tree. According to medical professionals, artifacts due to noise and thin topological changes are significant parameters among these criteria. These parameters can be determined via the Reeb graph, a topological descriptor commonly used in topological data analysis (TDA). In this article, we introduce a novel classification of saddle points, and a Reeb graph variant called the Local to Global Reeb graph (LGRG). We present parallel computation methods for critical points and LGRG, relying only on local information thanks to the homogeneity of the g-map formalism. We show that LGRG preserves the most subtle topological changes while simplifying the input into a graph formalism that respects the global structure of the mesh, allowing its use in future analyses.
We developed Beta Caller, an end-to-end system supporting the sport of rock climbing for climbers with visual impairment. Beta Caller provides real-time, audible instructions containing a prediction for the climber’s next move while they are actively climbing a rock wall. This system leverages computer vision techniques to collect key information about the climber’s environment, enabling Beta Caller to make move predictions on climbing walls it has never encountered before. Neural networks are used to predict where the climber should move next, based on information provided by the computer vision models. The predicted move is translated into a verbal message guiding the climber to the next hold and then transmitted via wireless headphones using a text-to-speech model. This novel idea makes one of the fastest growing sports in the world even more appealing and approachable to climbers with visual impairment, however, this tool can be utilized by all climbers to improve their climbing skills. Beta Caller achieved 80.08% accuracy predicting which limb the climber should move next and, when predicting the location of the next hold, Beta Caller achieved a bounding box error of only 6.79%. These results pioneer a strong foundation shaping the future landscape of rock climbing prediction tools for visually impaired climbers.
Lacunar fractal structures reduce the material quantity and weight while improving some physics properties, such as heat transfers, and preserving good mechanical properties. Nowadays, it is possible to construct such shapes thanks to additive manufacturing. This paper focuses on automatically generating subdivision rules for fractal lacunar structures with local topology control. The first main difficulty is guaranteeing topological consistency while assembling different cells to build a complicated multi-lacuna structure. The second is the adaptation of such shapes to geometric constraints like imposed boundaries. We address these questions throughout the formalism of the Boundary Controlled Iterated Function System. Then, we analyze the lacunarity and complexity of these structures from various geometric, topologic, and fractal measures.
