While the human visual system is known to be highly sensitive to global and configural shape information, deep neural networks models (DNNs) trained on ImageNet seem to favour local shape features. However, a more exact understanding of these differences has remained elusive, in part due to a lack of systematic methods for exploring the nature of high-dimensional shape representations. Here we argue that a novel shape frequency analysis can provide important insights into these representations. We explore this hypothesis through a series of experiments in which we measure human and DNN sensitivity across the shape frequency spectrum. These experiments reveal systematic differences between human and DNN models in spectral tuning and sensitivity to the amplitude and phase of shape frequency components. Using this frequency analysis approach, we also show that recent curriculum modifications claimed to create more human-like AI systems primarily act to attenuate sensitivity to higher shape frequencies rather than sharpening discriminative tuning to lower frequencies. Overall, we find that humans are tuned to much lower shape frequencies than DNN models, and as a consequence these models are unable to predict the majority of variance in human judgements of shape stimuli.

In team sports analytics, long-term player tracking remains a challenging task due to player appearance similarity, occlusion, and dynamic motion patterns. Accurately re-identifying players and reconnecting tracklets after extended absences from the field of view or prolonged occlusions is crucial for robust analysis. We introduce SportsSUSHI, a hierarchical graph-based approach that leverages domain-specific features, including jersey numbers, team IDs, and field coordinates, to enhance tracking accuracy. SportsSUSHI achieves high performance on the SoccerNet dataset and a newly proposed hockey tracking dataset. Our hockey dataset, recorded using a stationary camera capturing the entire playing surface, contains long sequences and annotations for team IDs and jersey numbers, making it well-suited for evaluating long-term tracking capabilities. The inclusion of domain-specific features in our approach significantly improves association accuracy, as demonstrated in our experiments. The dataset and code are available at https://github.com/mkoshkina/sports-SUSHI.

This paper discusses the results of the third edition of the Monocular Depth Estimation Challenge (MDEC). The challenge focuses on zero-shot generalization to the challenging SYNS-Patches dataset, featuring complex scenes in natural and indoor settings. As with the previous edition, methods can use any form of supervision, i.e. supervised or self-supervised. The challenge received a total of 19 submissions outperforming the baseline on the test set: 10 among them submitted a report describing their approach, highlighting a diffused use of foundational models such as Depth Anything at the core of their method. The challenge winners drastically improved 3D F-Score performance, from 17.51% to 23.72%.

Does the human visual system sample shapes at discrete points? During adaptation when the neurons are fatigued, one observes the underlying principles that were once less prominent than the fatigued features. Operating under deficit, these less prominent features expose the original contributions from the fatigued neurons that are now absent. An underlying lower order neural process, is thus, now revealed. In this paper, we conduct experiments using a modified version of the circle-polygon illusion in order to reveal the brain’s sampling pattern. The circle-polygon illusion produces polygonal percepts during adaptation when a static dark outline circle is pulsed at 2 Hz alternating with a gradient luminance circle. We perform two experiments. In the first experiment, we present circles of size [2,4,8,16] deg presented at eccentricity [0,1,2,4,8] deg in a cross design. In the second experiment, we modify the method of Sakurai (2014) and display arc lengths that are 1/8, 1/4, 3/8, 1/2, 5/8, 3/4, 7/8 and 1 (whole) of a circle, of size 4 and 8 deg, presented centrally. The observers report the edge length seen instead of a polygon order. We find that the stimulus size and presentation eccentricity, taken together, best explain the edge length reported by the users and that size alone is the most reasonable fixed parameter. The users, as a random effect, do not influence the mean of the edge length reported when considering the best model reported (size and eccentricity together) according to loglikelihood. However, the users do influence edge length reported only when using mean eccentricity or eccentricity as the parameter influencing edge length. Arc lengths of a circle produce same or similar edge lengths. The length of the curve does not play a significant role signifying that biological neurophysiology at an eccentricity controls the edge length formation.

Jersey number recognition is an important task in sports video analysis, partly due to its importance for long-term player tracking. It can be viewed as a variant of scene text recognition. However, there is a lack of published attempts to apply scene text recognition models on jersey number data. Here we introduce a novel public jersey number recognition dataset for hockey and study how scene text recognition methods can be adapted to this problem. We address issues of occlusions and assess the degree to which training on one sport (hockey) can be generalized to another (soccer). For the latter, we also consider how jersey number recognition at the single-image level can be aggregated across frames to yield tracklet-level jersey number labels. We demonstrate high performance on image- and tracklet-level tasks, achieving 91.4% accuracy for hockey images and 87.4% for soccer tracklets. Code, models, and data are available at https://github.com/mkoshkina/jerseynumber-pipeline.

Semantic segmentation is an important sub-task for many applications. However, pixel-level ground-truth labeling is costly, and there is a tendency to overfit to training data, thereby limiting the generalization ability. Unsupervised domain adaptation can potentially address these problems by allowing systems trained on labelled datasets from the source domain (including less expensive synthetic domain) to be adapted to a novel target domain. The conventional approach involves automatic extraction and alignment of the representations of source and target domains globally. One limitation of this approach is that it tends to neglect the differences between classes: representations of certain classes can be more easily extracted and aligned between the source and target domains than others, limiting the adaptation over all classes. Here, we address this problem by introducing a Class-Conditional Domain Adaptation (CCDA) method. This incorporates a class-conditional multi-scale discriminator and class-conditional losses for both segmentation and adaptation. Together, they measure the segmentation, shift the domain in a class-conditional manner, and equalize the loss over classes. Experimental results demonstrate that the performance of our CCDA method matches, and in some cases, surpasses that of state-of-the-art methods.

Accurate turning movement counts at intersections are important for signal control, traffic management and urban planning. Computer vision systems for automatic turning movement counts typically rely on visual analysis in the image plane of an infrastructure camera. Here we explore potential advantages of back-projecting vehicles detected in one or more infrastructure cameras to the ground plane for analysis in real-world 3D coordinates. For single-camera systems we find that back-projection yields more accurate trajectory classification and turning movement counts. We further show that even higher accuracy can be achieved through weak fusion of back-projected detections from multiple cameras. These results suggeest that traffic should be analyzed on the ground plane, not the image plane.

BACKGROUND. The statistics of coarse and fine shape features can be analyzed using a Fourier descriptor projection. Natural shapes are known to be lowpass: coarse shape features (low shape frequencies) typically have higher amplitudes than fine shape features (high frequencies). Prior work suggests that human shape sensitivity is even more biased toward low shape frequencies than is optimal for natural lowpass shapes, however this was demonstrated only for a simple binary shape discrimination task within a linear classification framework. Deep networks are reported to be more sensitive to local shape features, suggesting a high-frequency bias, but these demonstrations have primarily been on simple artificial stimuli. Here we employ a novel Fourier method to assess the processing of coarse and fine shape features of natural shapes by humans and deep networks, in a more realistic object classification task. METHOD. Human observers (n = 11) classified frequency-filtered animal silhouettes into one of nine animal categories. To assess sensitivity to shape frequencies, the stimuli were high-pass filtered to progressively remove the lowest shape frequencies, with cutoffs ranging from the 2nd to 8th harmonic. Two representative deep networks were also evaluated on the same stimuli: a convolutional network (ResNet-50) and a transformer network (ViT). RESULTS. Both human and deep network performance declined rapidly as low shape frequencies were progressively eliminated. Trial-by-trial analysis revealed that ViT is more predictive of human responses than ResNet-50. Interestingly, the proportion of explainable human variance accounted for by ViT increased from 29% to 57% as more of the low frequencies were eliminated, suggesting that while this transformer model captures some aspects of human selectivity for higher shape frequencies, it struggles to account for human processing of the lower shape frequencies that largely determine human shape judgements.

Objective. Human observers can appreciate 3D properties such as surface attitude in 2D images. Here we evaluate methods for making surface attitude judgements for real-world scenes and the effect of context on such judgements. Methods: Real-world image patches from the SYNS dataset (Adams et al., 2016) were presented monocularly. In Experiment 1, we tested two response methods for judging 9° image patches: a gauge figure providing perspective cues and a 2D dial with separate slant and tilt indicators. In Experiment 2, we used the gauge figure method to compare judgments of patches with two different sizes: a 9° diameter circle presented full size and 64° by 36° patch presented at 37.5% of full size. The latter provided more contextual visual information. Results: Slant: Slant judgements from the gauge figure method correlated more strongly with the ground truth than the dial method and was reported to be the easier method to use. The larger patches (Exp 2) also produced a stronger correlation with ground truth than the smaller patches. Tilt: Some of our observers produced a significant correlation between tilt judgements and ground truth for the small image patches of Experiment 1. However, observers often mischaracterised many real-world tilt values as facing left or right, although cardinal facing surfaces were judged correctly. Preliminary data for Experiment 2 showed improved tilt judgements for the larger patches although the same mischaracterisation persisted. Conclusion: Both the gauge figure and dial methods can produce reasonable judgements of slant and tilt for real-world images, but the dial method is less reliable and more difficult to use. Tilt judgements for real-world images indicated a potential bias toward allocentric (gravity-centered) coordinates. While providing a wider field of view generally improves judgements, it does not eliminate this allocentric bias.

The local shape of the occluding contour of an object is known to constrain the local shape of the object surface [1], however, these known constraints are qualitative. Here we posit here that in addition to these qualitative constraints, typical regularities of common objects and rules of projection induce dependencies that can be used to derive statistical estimates of quantitative solid shape from the occluding contour. To explore this conjecture, we partition the problem into two parts: 1) Estimation of the 3D rim from the 2D occluding contour, and 2) Estimation of the visible surface shape from the estimated 3D rim. We train and evaluate a number of statistical models on two distinct 3D object datasets and demonstrate that capturing these statistical regularities leads to better estimates of 3D shape than existing methods.

We describe and demonstrate a sensorimotor architecture for social robots, highlighting in particular the value of attentive sensing for acquisition of more detailed information about people in the far field, and how this can inform the robot’s behaviour.

Objective. Our understanding of surface attitude perception derives largely from studies using synthetic stimuli and gauge figure probes. Here we assess whether these studies generalize to real-world scenes and whether misperception of the gauge figure could lead to bias in these results. Methods: Stimuli were viewed monocularly through a 12° circular aperture. We explored two classes of stimuli and two modes of response. Stimuli: 1) Planar surfaces painted with synthetic textures; 2) Real-world patches from the SYNS dataset (Adams et al., 2016). Response mode: 1) gauge figure; 2) 2D dial with separate slant and tilt indicators. In addition we tested the synthetic textures binocularly with perspective only and with disparity; and compared 2D and 3D gauge figure and dial methods. Results: Slant: Observers made meaningful judgements of slant for both synthetic and real-world surfaces. For synthetic textures, RMS error and sensitivity (JND) were similar for the two methods. While the dial method produced a gain greater than 1, the gauge figure method produced a gain less than 1 and was biased to overestimate slant, possibly due to underestimation of the slant of the gauge figure itself. For real-world surfaces, in contrast, both response modes produced a negative bias: observers underestimated slant. Tilt: For synthetic textures, tilt judgements were precise and accurate for both methods. However, observers were unable to perform the task for real-world surfaces, instead reporting tilts tightly clustered around the cardinal directions. Conclusion: While both gauge and dial methods work well for tilt judgements of synthetic surfaces, for slant judgements both methods generate strong biases, and it remains unclear whether this reflects biased perception of the surfaces, biases in the method, or both. Humans appear to be unable to make reasonable judgements of egocentric tilt for natural surfaces.

Most existing super-resolution (SR) methods are designed to restore high resolution (HR) images from certain low resolution (LR) images with a simple degradation, e.g. bicubic downsampling. Their generalization capability to real-world degradation is limited because it often couples several degradation factors such as noise and blur. To solve this problem, existing blind SR methods rely on either explicit degradation estimation or translation to bicubicly downsampled LR images, where inaccurate estimation or translation would severely deteriorate the SR performance. In this paper, we propose a plug-and-play module, which could be applied to any existing image super-resolution model for feature-level adaptation to improve the generalization ability to real-world degraded images. Specifically, a degradation encoder is proposed to compute an implicit degradation representation with a ranking loss based on the degradation level as supervision. The degradation representation then works as a kind of condition and is applied to the existing image super-resolution model pretrained on bicubicly downsampled LR images through the proposed region-aware modulation. With the proposed method, the base super-resolution model could be fine-tuned to adapt to the condition of degradation representation for further improvement. Experimental results on both synthetic and real-world datasets show that the proposed image SR method with compact model size performs favorably against state-of-the-art methods. Our source code is publicly available at https://github.com/wangyue7777/blindsr_daa .

This paper summarizes the results of the first Monocular Depth Estimation Challenge (MDEC) organized at WACV2023. This challenge evaluated the progress of self-supervised monocular depth estimation on the challenging SYNS-Patches dataset. The challenge was organized on CodaLab and received submissions from 4 valid teams. Participants were provided a devkit containing updated reference implementations for 16 State-of-the-Art algorithms and 4 novel techniques. The threshold for acceptance for novel techniques was to outperform every one of the 16 SotA baselines. All participants outperformed the baseline in traditional metrics such as MAE or AbsRel. However, pointcloud reconstruction metrics were challenging to improve upon. We found predictions were characterized by interpolation artefacts at object boundaries and errors in relative object positioning. We hope this challenge is a valuable contribution to the community and encourage authors to participate in future editions.