Despite recent progress in 3D self-supervised learning, collecting large-scale 3D scene scans remains expensive and labor-intensive. In this work, we investigate whether 3D representations can be learned from unlabeled videos recorded without any real 3D sensors. We present Laplacian-Aware Multi-level 3D Clustering with Sinkhorn-Knopp (LAM3C), a self-supervised framework that learns from video-generated point clouds from unlabeled videos. We first introduce \data, a video-generated point cloud dataset constructed by collecting room-walkthrough videos from the web (e.g., real-estate tours) and generating 49,219 scenes using an off-the-shelf feed-forward reconstruction model. We also propose a noise-regularized loss that stabilizes representation learning by enforcing local geometric smoothness and ensuring feature stability under noisy point clouds. Remarkably, without using any real 3D scans, LAM3C achieves higher performance than the previous self-supervised methods on indoor semantic and instance segmentation. These results suggest that unlabeled videos represent an abundant source of data for 3D self-supervised learning.

Zero-shot recognition models require extensive training data for generalization. However, in zero-shot 3D classification, collecting 3D data and captions is costly and laborintensive, posing a significant barrier compared to 2D vision. Recent advances in generative models have achieved unprecedented realism in synthetic data production, and recent research shows the potential for using generated data as training data. Here, naturally raising the question: Can synthetic 3D data generated by generative models be used as expanding limited 3D datasets? In response, we present a synthetic 3D dataset expansion method, Textguided Geometric Augmentation (TeGA). TeGA is tailored for language-image-3D pretraining, which achieves SoTA in zero-shot 3D classification, and uses a generative textto-3D model to enhance and extend limited 3D datasets. Specifically, we automatically generate text-guided synthetic 3D data and introduce a consistency filtering strategy to discard noisy samples where semantics and geometric shapes do not match with text. In the experiment to double the original dataset size using TeGA, our approach demonstrates improvements over the baselines, achieving zeroshot performance gains of 3.0 These results demonstrate that TeGA effectively bridges the 3D data gap, enabling robust zero-shot 3D classification even with limited real training data and paving the way for zero-shot 3D vision application.

In the recent years, the research community has witnessed growing use of 3D point cloud data for the high applicability in various real-world applications. By means of 3D point cloud, this modality enables to consider the actual size and spatial understanding. The applied fields include mechanical control of robots, vehicles, or other real-world systems. Along this line, we would like to improve 3D point cloud instance segmentation which has emerged as a particularly promising approach for these applications. However, the creation of 3D point cloud datasets entails enormous costs compared to 2D image datasets. To train a model of 3D point cloud instance segmentation, it is necessary not only to assign categories but also to provide detailed annotations for each point in the large-scale 3D space. Meanwhile, the increase of recent proposals for generative models in 3D domain has spurred proposals for using a generative model to create 3D point cloud data. In this work, we propose a pre-training with 3D synthetic data to train a 3D point cloud instance segmentation model based on generative model for 3D scenes represented by point cloud data. We directly generate 3D point cloud data with Point-E for inserting a generated data into a 3D scene. More recently in 2025, although there are other accurate 3D generation models, even using the Point-E as an early 3D generative model can effectively support the pre-training with 3D synthetic data. In the experimental section, we compare our pre-training method with baseline methods indicated improved performance, demonstrating the efficacy of 3D generative models for 3D point cloud instance segmentation.

Vision Foundation Models (VFMs) have made remarkable progress and are increasingly being applied to segmentation tasks in real-world industrial settings. However, VFMs pre-trained on real-image datasets still face several challenges: (1) they do not always perform well on industrial datasets due to significant differences from natural imagery, (2) legal and ethical restrictions, such as limitations on commercial use, constrain extensibility, and (3) building training frameworks under limited computational and data resources remains a critical issue. These challenges raise a fundamental question: can we construct industrial segmentation models without relying on real images or manual annotations? To address this question, we propose the Instance Core Segment Dataset (InsCore), a synthetic data generation framework and the resulting pre-training dataset based on Formula-Driven Supervised Learning (FDSL). InsCore is designed not around the visual appearance or domain of real images, but around the hypothesis that learning to handle complex occlusions during pre-training is a key factor for strong performance in industrial domains. Through experiments across five domains (medical, biomedical, remote sensing, manufacturing, and logistics) we demonstrate that InsCore pre-trained models achieve average mAP scores of 45.2 with the ViTDet backbone and 46.0 with the Swin Transformer backbone, on par with ImageNet-21k supervised pre-training (45.0) while using no real images at all. As a reference point under different input assumptions, prompted SAM with ground-truth bounding boxes attains 45.4 on the same benchmarks. Finally, InsCore consists of only 100k images and 3.2M masks, roughly 1/110 and 1/312 the scale of the SA-1B dataset.

In this work, we investigate the understudied effect of the training data used for image super-resolution (SR). Most commonly, novel SR methods are developed and benchmarked on common training datasets such as DIV2K and DF2K. However, we investigate and rethink the training data from the perspectives of diversity and quality, thereby addressing the question of "How important is SR training for SR models?". To this end, we propose an automated image evaluation pipeline. With this, we stratify existing high-resolution image datasets and larger-scale image datasets such as ImageNet and PASS to compare their performances. We find that datasets with (i) low compression artifacts, (ii) high within-image diversity as judged by the number of different objects, and (iii) a large number of images from ImageNet or PASS all positively affect SR performance. We hope that the proposed simple-yet-effective dataset curation pipeline will inform the construction of SR datasets in the future and yield overall better models.

Throughout the history of computer vision, while research has explored the integration of images (visual) and point clouds (geometric), many advancements in image and 3D object recognition have tended to process these modalities separately. We aim to bridge this divide by integrating images and point clouds on a unified transformer model. This approach integrates the modality-specific properties of images and point clouds and achieves fundamental downstream tasks in image and 3D object recognition on a unified transformer model by learning visual-geometric representations. In this work, we introduce Formula-Supervised Visual-Geometric Pre-training (FSVGP), a novel synthetic pre-training method that automatically generates aligned synthetic images and point clouds from mathematical formulas. Through cross-modality supervision, we enable supervised pre-training between visual and geometric modalities. FSVGP also reduces reliance on real data collection, cross-modality alignment, and human annotation. Our experimental results show that FSVGP pre-trains more effectively than VisualAtom and PC-FractalDB across six tasks: image and 3D object classification, detection, and segmentation. These achievements demonstrate FSVGP's superior generalization in image and 3D object recognition and underscore the potential of synthetic pre-training in visua-lgeometric representation learning. Our project website is available at https://ryosuke-yamada.github.io/fdsl-fsvgp/.

Pre-training for 3D object recognition typically requires a large-scale 3D dataset to learn effective 3D geometric representations. However, constructing such datasets is costly due to the extensive 3D data collection and human annotation required. This paper explores a synthetic pre-training approach that learns 3D geometric representations by reconstructing structural point clouds without relying on real data or human annotation. We propose the Point Cloud Perlin Noise (PCPN) dataset, which is an automatically generated point cloud dataset that uses Perlin noise to simulate natural 3D structures found in the real world. The proposed method enables the rapid generation of diverse 3D geometric patterns using a simple Perlin noise-based formula, significantly reducing the human effort typically involved in creating conventional 3D datasets. We applied PointMAE to the PCPN dataset for pre-training, demonstrating improved performance in downstream tasks such as 3D shape classification and part segmentation. Our experiments showed that the proposed pre-trained model outperformed a model trained from scratch on ModelNet40 by 1.4%. In addition, our pre-training strategy proves effective for 3D object recognition without requiring real data or supervised labels. This study highlights that Perlin noise can capture 3D structural properties and that the diversity of geometric patterns is crucial for learning effective 3D geometric representations.

Pre-training and transfer learning are an important building block of current computer vision systems. While pre-training is usually performed on large real-world image datasets, in this paper we ask whether this is truly necessary. To this end, we search for a minimal, purely synthetic pre-training dataset that allows us to achieve performance similar to the 1 million images of ImageNet-1k. We construct such a dataset from a single fractal with perturbations. With this, we contribute three main findings. (i) We show that pre-training is effective even with minimal synthetic images, with performance on par with large-scale pre-training datasets like ImageNet-1k for full fine-tuning. (ii) We investigate the single parameter with which we construct artificial categories for our dataset. We find that while the shape differences can be indistinguishable to humans, they are crucial for obtaining strong performances. (iii) Finally, we investigate the minimal requirements for successful pre-training. Surprisingly, we find that a substantial reduction of synthetic images from 1k to 1 can even lead to an increase in pre-training performance, a motivation to further investigate “scaling backwards”. Finally, we extend our method from synthetic images to real images to see if a single real image can show similar pre-training effect through shape augmentation. We find that the use of grayscale images and affine transformations allows even real images to “scale backwards”. The code is available at https://github.com/SUPER-TADORY/1p-frac .

The construction of 3D medical image datasets presents several issues, including requiring significant financial costs in data collection and specialized expertise for annotation, as well as strict privacy concerns for patient confidentiality compared to natural image datasets. Therefore, it has become a pressing issue in 3D medical image segmentation to enable data-efficient learning with limited 3D medical data and supervision. A promising approach is pre-training, but improving its performance in 3D medical image segmentation is difficult due to the small size of existing 3D medical image datasets. We thus present the Primitive Geometry Segment Pre-training (PrimGeoSeg) method to enable the learning of 3D semantic features by pre-training segmentation tasks using only primitive geometric objects for 3D medical image segmentation. PrimGeoSeg performs more accurate and efficient 3D medical image segmentation without manual data collection and annotation. Further, experimental results show that PrimGeoSeg on SwinUNETR improves performance over learning from scratch on BTCV, MSD (Task06), and BraTS datasets by 3.7 equal to or better than state-of-the-art self-supervised learning despite the equal number of pre-training data. From experimental results, we conclude that effective pre-training can be achieved by looking at primitive geometric objects only. Code and dataset are available at https://github.com/SUPER-TADORY/PrimGeoSeg.

Daily indoor scenes often involve constant changes due to human activities. To recognize scene changes, existing change captioning methods focus on describing changes from two images of a scene. However, to accurately perceive and appropriately evaluate physical changes and then identify the geometry of changed objects, recognizing and localizing changes in 3D space is crucial. Therefore, we propose a task to explicitly localize changes in 3D bounding boxes from two point clouds and describe detailed scene changes, including change types, object attributes, and spatial locations. Moreover, we create a simulated dataset with various scenes, allowing generating data without labor costs. We further propose a framework that allows different 3D object detectors to be incorporated in the change detection process, after which captions are generated based on the correlations of different change regions. The proposed framework achieves promising results in both change detection and captioning. Furthermore, we also evaluated on data collected from real scenes. The experiments show that pretraining on the proposed dataset increases the change detection accuracy by +12.8% (mAP0.25) when applied to real-world data. We believe that our proposed dataset and discussion could provide both a new benchmark and in-sights for future studies in scene change understanding.

Although we have witnessed Neural Architecture Search (NAS), which automatically explores architecture for best performance, the discussion has not advanced considering a dataset. We discuss the potential of Neural Dataset Search (NDS), which explores the appropriate configuration in a pre-training dataset to achieve a better pre-training effect. The NDS is designed to train in order to find the optimal parameters in the pre-training dataset for a given network architecture and downstream tasks. This allows for predicting the optimal pre-training parameters for a new unseen task in one shot. Thus, the NDS has the potential to bottom up the effectiveness of the pre-training. Therefore, this paper focuses on formula-driven supervised learning, and as a first consideration, we verify the appropriate configuration in Residual Network (ResNet) and Fractal DataBase (FractalDB). From the experimental results, we confirmed that the FractalDB generation parameters that provide the best pre-training effect are different for each ResNet-{18, 50, 152}. These observations reveal that there is an adapted image representation or dataset structure (e.g., input size, parameter, category) for a particular architecture. We hope these results will encourage further research on NDS that fully exploits the pre-training of synthetic images.

In 3D medical image segmentation, data collection and annotation costs require significant human efforts. Moreover, obtaining training data is challenging due to privacy constraints. Consequently, achieving efficient learning with limited data is an urgent 3D medical image segmentation issue. One approach to address this problem is using pre-trained models, which have been widely researched. Recently, self-supervised learning for 3D medical images has gained popularity, but the data available for such learning is also scarce, limiting the number of pre-training datasets. In recent years, formula-driven supervised learning has garnered attention. It can achieve high pre-training effects using only easily accessible synthetic data, making it a promising alternative for pre-training datasets. Inspired by this approach, we propose the Auto-generated Volumetric Shapes Database (AVS-DB) for data-scarce 3D medical image segmentation tasks. AVS-DB is automatically generated from a combination of dozens of 3D models based on polygons and shape similarity ratio variations. Our experiments show that AVS-DB pre-trained models significantly outperform models trained from scratch and achieve comparable or better performance than existing self-supervised learning methods we compared. AVS-DB can potentially enhance 3D medical image segmentation models and address limited data availability challenges.

Why is there disparity in the miss rates of pedestrian detection between different age attributes? In this study, we propose to (i) improve the accuracy of pedestrian detection using our pre-trained model; and (ii) explore the causes of this disparity. In order to improve detection accuracy, we extend a pedestrian detection pre-training dataset, the Weakly Supervised Pedestrian Dataset (WSPD), by means of self-training, to construct our Self-Trained Person Dataset (STPD). Moreover, we hypothesize that the cause of the miss rate is due to three biases: (1) the apparent bias towards "adults" versus "children"; (2) the quantity of training data bias against "children"; and (3) the scale bias of the bounding box. In addition, we constructed an evaluation dataset by manually annotating "adult" and "child" bounding boxes to the INRIA Person Dataset. As a result, we confirm that the miss rate was reduced by up to 0.4% for adults and up to 3.9% for children. In addition, we discuss the impact of the size and appearance of the bounding boxes on the disparity in miss rates and provide an outlook for future research.

The construction of 3D point cloud datasets requires a great deal of human effort. Therefore, constructing a large-scale 3D point clouds dataset is difficult. In order to rem-edy this issue, we propose a newly developed point cloud fractal database (PC-FractalDB), which is a novel family of formula-driven supervised learning inspired by fractal geometry encountered in natural 3D structures. Our re-search is based on the hypothesis that we could learn rep-resentations from more real-world 3D patterns than con-ventional 3D datasets by learning fractal geometry. We show how the PC-FractalDB facilitates solving several re-cent dataset-related problems in 3D scene understanding, such as 3D model collection and labor-intensive annotation. The experimental section shows how we achieved the performance rate of up to 61.9% and 59.0% for the Scan-NetV2 and SUN RGB-D datasets, respectively, over the current highest scores obtained with the PointContrast, con-trastive scene contexts (CSC), and RandomRooms. More-over, the PC-FractalDB pre-trained model is especially ef-fective in training with limited data. For example, in 10% of training data on ScanNetV2, the PC-FractalDB pre-trained VoteNet performs at 38.3%, which is +14.8% higher accu-racy than CSC. Of particular note, we found that the pro-posed method achieves the highest results for 3D object de-tection pre-training in limited point cloud data. 1 1 Dataset release: https://ryosuke-yamada.github.io/PointCloud-FractalDataBase/

In the present work, we show that the performance of formula-driven supervised learning (FDSL) can match or even exceed that of ImageNet-21k without the use of real images, human-, and self-supervision during the pre-training of Vision Transformers (ViTs). For example, ViT-Base pre-trained on ImageNet-21k shows 81.8% top-1 accuracy when fine-tuned on ImageNet-1k and FDSL shows 82.7% top-1 accuracy when pre-trained under the same conditions (number of images, hyperparameters, and number of epochs). Images generated by formulas avoid the privacy/copyright issues, labeling cost and errors, and biases that real images suffer from, and thus have tremendous potential for pre-training general models. To understand the performance of the synthetic images, we tested two hypotheses, namely (i) object contours are what matter in FDSL datasets and (ii) increased number of parameters to create labels affects performance improvement in FDSL pre-training. To test the former hypothesis, we constructed a dataset that consisted of simple object contour combinations. We found that this dataset can match the performance of fractals. For the latter hypothesis, we found that increasing the difficulty of the pre-training task generally leads to better fine-tuning accuracy.

Is it possible to use convolutional neural networks pre-trained without any natural images to assist natural image understanding? The paper proposes a novel concept, Formula-driven Supervised Learning. We automatically generate image patterns and their category labels by assigning fractals, which are based on a natural law existing in the background knowledge of the real world. Theoretically, the use of automatically generated images instead of natural images in the pre-training phase allows us to generate an infinite scale dataset of labeled images. Although the models pre-trained with the proposed Fractal DataBase (FractalDB), a database without natural images, does not necessarily outperform models pre-trained with human annotated datasets at all settings, we are able to partially surpass the accuracy of ImageNet/Places pre-trained models. The image representation with the proposed FractalDB captures a unique feature in the visualization of convolutional layers and attentions.

We propose the generation method of large-scale image database for pre-training in 3D object recognition. The method is inspired from the principles of nature law. We adopt fractal geometry to represent the principles and build the Fractal Data Base random search (FractalDBrs). In contrast to traditional image database such as ImageNet, Iterated Function System (IFS) automatically generates large amount of image data to build the proposed FractalDBrs in short time without menial labors such as collecting and annotating images. In the experiments, we utilized the FractalDBrs and traditional databases; ImageNet, CIFAR100, Caltech256, or Places365, for pre-training in 3D object recognition with ModelNet40. The model pre-trained with FractalDBrs achieved the highest discrimination accuracy of 97.12% against the second highest accuracy of 96.43% with ImageNet. For reference, the model trained from scratch achieved 91.53% discrimination accuracy. We have verified the effectiveness of the proposed fractal geometry-based image database for pre-training in 3D object recognition.

The paper proposes a method for automatic multi-view dataset construction based on formula-driven supervised learning (FDSL). Although data collection and human annotation of 3D objects are labor-intensive, we automatically generate their training data and labels in the proposed multi-view dataset. To create a large-scale multi-view dataset, we employ fractal geometry, which is considered the background information of many objects in the real world. We project in a circle from the rendered 3D fractal models to construct the Multi-view Fractal DataBase (MV-FractalDB), which is then used to make a pre-trained CNN model. According to the experimental results, the MV-FractalDB pre-trained model surpasses the accuracies with self-supervised methods (e.g., SimCLR and MoCo) and is close to supervised methods (e.g., ImageNet) in terms of performance rates on multi-view image datasets. We demonstrate the potential of FDSL for multi-view image recognition.

Large-scale image datasets are constructed with a significant amount of time and human effort, however, an image may have biases in the camera positions that render the target object. In this study, we propose a framework that automatically conducts 3D models, multi-view images, and even category definition. We automatically generate 3D models based on fractal geometry, which is the regularity behind natural phenomena and render images from multiple viewpoints. By following those generation processes, we can automatically construct a generally large-scale image dataset that takes into account the viewpoint position. The experimental results show that the classification accuracy is improved over the conventional baseline in the context of pre-training for image recognition tasks. We show that our proposed method provides an effective method for automatically constructing pre-training datasets for image recognition tasks.