Vision Language Models (VLMs) are designed to extend Large Language Models (LLMs) with visual capabilities, yet in this work we observe a surprising phenomenon: VLMs can outperform their underlying LLMs on purely text-only tasks, particularly in long-context information retrieval. To investigate this effect, we build a controlled synthetic retrieval task and find that a transformer trained only on text achieves perfect in-distribution accuracy but fails to generalize out of distribution, while subsequent training on an image-tokenized version of the same task nearly doubles text-only OOD performance. Mechanistic interpretability reveals that visual training changes the model’s internal binding strategy: text-only training encourages positional shortcuts, whereas image-based training disrupts them through spatial translation invariance, forcing the model to adopt a more robust symbolic binding mechanism that persists even after text-only examples are reintroduced. We further characterize how binding strategies vary across training regimes, visual encoders, and initializations, and show that analogous shifts occur during pretrained LLM-to-VLM transitions. Our findings suggest that cross‑modal training can enhance reasoning and generalization even for tasks grounded in a single modality.

Extracting key information from news articles is crucial for advancing search systems. Historically, the 5W1H framework, which organises information based on 'Who', 'What', 'When', 'Where', 'Why', and 'How', has been a predominant method in digital journalism empowering search tools. The rise of Large Language Models (LLMs) has sparked new research into their potential for performing such information extraction tasks effectively. Our study examines a novel approach to employing LLMs in the 5W1H extraction process, particularly focusing on their capacity to mimic human reasoning. We introduce two innovative Chain-of-Thought (COT) prompting techniques to extract 5W1H in news: extractive reasoning and question-level reasoning. The former directs the LLM to pinpoint and highlight essential details from texts, while the latter encourages the model to emulate human-like reasoning at the question-response level. Our research methodology includes experiments with leading LLMs using prompting strategies to ascertain the most effective approach. The results indicate that COT prompting significantly outperforms other methods. In addition, we show that the effectiveness of LLMs in such tasks depends greatly on the nature of the questions posed.

During the last few years, the field of dynamical systems has been developing innovative tools to study the asymptotic behavior of different optimizers in the context of neural networks. In this work, we redefine an extensively studied optimizer, employing classical techniques from hyperbolic geometry. This new definition is linked to a non-linear differential equation as a continuous limit. Additionally, by utilizing Lyapunov stability concepts, we analyze the asymptotic behavior of its critical points.

Recognizing variable stars is a task of interest in the astronomy community. Currently, this task has taken advantage of deep learning algorithms. However, these algorithms require a large amount of data to achieve high levels of precision. In this work, self-supervised learning is proposed to improve the classification of variable stars considering a reduced amount of data using recurrent networks. The experiments in Gaia dataset show that the proposed approach allows to improve performance, when compared with traditional initialization schemes, up to 7% and 13% in real databases in semi-supervised learning scenarios. In future work, we propose considering experiments with other variable star databases.

This paper introduces SpatialCluster, a Python library developed for clustering urban areas using geolocated data. The library integrates a range of methods for urban clustering, including Deep Modularity Networks, Gaussian Mixtures, K-Nearest Neighbours, Self Organized Maps, and Information-Theoretic Clustering, providing a comprehensive framework. These methods are evaluated using indices such as the Adjusted Rand Index and Adjusted Mutual Information, and the library includes features for detailed map visualization. SpatialCluster’s online documentation offers examples, making the library accessible to researchers and urban planners. The library aims to facilitate urban data analysis and contribute to the field of urban studies.

News media outlets disseminate information across various platforms. Often, these posts present complementary content and perspectives on the same news story. However, to compile a set of related news articles, users must thoroughly scour multiple sources and platforms, manually identifying which publications pertain to the same story. This tedious process hinders the speed at which journalists can perform essential tasks, notably fact-checking. To tackle this problem, we created a dataset containing both related and unrelated news pairs. This dataset allows us to develop information retrieval models grounded in the principle of binary relevance. Recognizing that many Transformer-based models might be suited for this task but could overemphasize relationships based on lexical connections, we tailored a dataset to fine-tune these models to focus on semantically relevant connections in the news domain. To craft this dataset, we introduced a methodology to identify pairs of news stories that are lexically similar yet refer to different events and pairs that discuss the same event but have distinct lexical structures. This design compels Transformers to recognize semantic connections between stories, even when their lexical similarities might be absent. Following a human-annotation assessment, we reveal that BERT outperformed other techniques, excelling even in challenging test cases. To ensure the reproducibility of our approach, we have made the dataset and top-performing models publicly available.

The way in which public transport buses are driven has an influence in users’perception and satisfaction with the service. Bus driver’s behavior is usually obtained surveying passengers and/or using the mystery passenger method, not necessarily allowing for an objective and continuous evaluation. In this work, we introduce a novel methodology to automatically classify drivers’ behavior in a more consistent and objective manner, based on data from inertial measurement units, and machine learning techniques. By substituting human evaluators with automatic data collection and classification algorithms, we are able to reduce the subjectivity and cost of the current methodology, while increasing sample size. Our approach is based on three components: i) data capture using inertial measurement units (e.g. mobile devices), ii) carefully tuned classifiers that deal with sample imbalance problems, and iii) an interpretable scoring system. Results show that collected data captures several types of undesirable maneuvers, providing a rich information to the classification process. In terms of categorization performance, the evaluated classifiers, namely support vector machines, decision trees and k-NN, deliver high and consistent accuracy after the tuning process, even in the presence of a highly imbalanced sample. Finally, the proposed driver’s behavior score shows high discriminative power, effectively characterizing differences between drivers, and providing driver-tailored driving recommendations, that can be generated in specific spots, in order to improve passengers’ experience. The resulting methodology can be cost-effectively deployed at a large scale with good performance.

The COVID-19 pandemic has underlined the need for reliable information for clinical decision-making and public health policies. As such, evidence-based medicine (EBM) is essential in identifying and evaluating scientific documents pertinent to novel diseases, and the accurate classification of biomedical text is integral to this process. Given this context, we introduce a comprehensive, curated dataset composed of COVID-19-related documents.This dataset includes 20,047 labeled documents that were meticulously classified into five distinct categories: systematic reviews (SR), primary study randomized controlled trials (PS-RCT), primary study non-randomized controlled trials (PS-NRCT), broad synthesis (BS), and excluded (EXC). The documents, labeled by collaborators from the Epistemonikos Foundation, incorporate information such as document type, title, abstract, and metadata, including PubMed id, authors, journal, and publication date.Uniquely, this dataset has been curated by the Epistemonikos Foundation and is not readily accessible through conventional web-scraping methods, thereby attesting to its distinctive value in this field of research. In addition to this, the dataset also includes a vast evidence repository comprising 427,870 non-COVID-19 documents, also categorized into SR, PS-RCT, PS-NRCT, BS, and EXC. This additional collection can serve as a valuable benchmark for subsequent research. The comprehensive nature of this open-access dataset and its accompanying resources is poised to significantly advance evidence-based medicine and facilitate further research in the domain.

Urban clustering detects geographical units that are internally homogeneous and distinct from their surroundings. It has applications in urban planning, but few studies compare the effectiveness of different methods. We study two techniques that represent two families of urban clustering algorithms: Gaussian Mixture Models (GMMs), which operate on spatially distributed data, and Deep Modularity Networks (DMONs), which work on attributed graphs of proximal nodes. To explore the strengths and limitations of these techniques, we studied their parametric sensitivity under different conditions, considering the spatial resolution, granularity of representation, and the number of descriptive attributes, among other relevant factors. To validate the methods, we asked residents of Santiago, Chile, to respond to a survey comparing city clustering solutions produced using the different methods. Our study shows that DMON is slightly preferred over GMM and that social features seem to be the most important ones to cluster urban areas.

In the last decade, large street imagery data sets and machine learning developments have allowed increasing scalability of methodologies to understand the effects of landscape attributes on the way they are perceived. However, these new methodologies have not incorporated individual heterogeneity in their analysis, even though differences by gender and other sociodemographic characteristics in the perception of safety and other aspects of landscapes and public spaces have been widely studied in social sciences and urban planning in lower scale studies. In the present study, we combine computational and statistical tools to develop a methodological proposal with high scalability and low implementation cost, which helps to identify and measure heterogeneous perception and its correlation to the presence of elements in the landscape. To achieve this, we implement a survey of perception of public spaces, collecting sociodemographic information of respondents. Then, we fit a discrete choice model to quantify perceptions of these spaces using a parametrization of images that jointly considers semantic segmentation and object detection as input. Our results show heterogeneity in the perception of safety in public spaces according to gender and the observer’s habitual mobility choices. The model is then applied to the city of Santiago, Chile. This produces a map of safety perception for different types of users. The proposed method and the obtained results can be a relevant input for the design of public spaces and decision making in the urban planning process.

Electrolytic refining is the last step of pyrometallurgical copper production.Here, smelted copper is converted into high-quality cathodes through electrolysis.Cathodes that do not meet the physical quality standards are rejected and further reprocessed or sold at a minimum profit.Prediction of cathodic rejection is therefore of utmost importance to accurately forecast the electrorefining cycle economic production.Several attempts have been made to estimate this process outcomes, mostly based on physical models of the underlying electrochemical reactions.However, they do not stand the complexity of real operations.Data-driven methods, such as deep learning, allow modeling complex non-linear processes by learning representations directly from the data.We study the use of several recurrent neural network models to estimate the cathodic rejection of a cathodic cycle, using a series of operational measurements throughout the process.We provide an ARMAX model as a benchmark.Basic recurrent neural network models are analyzed first: a vanilla RNN and an LSTM model provide an initial approach.These are further composed into an Encoder-Decoder model, that uses an attention mechanism to selectively weight the input steps that provide most information upon inference.This model obtains 5.45% relative error, improving by 81.4% the proposed benchmark.Finally, we study the attention mechanism's output to distinguish the most relevant electrorefining process steps.We identify the initial state as critical in predicting cathodic rejection.This information can be used as an input for decision support systems or control strategies to reduce cathodic rejection and improve electrolytic refining's profitability.

Continuous learning occurs naturally in human beings.However, Deep Learning methods suffer from a problem known as Catastrophic Forgetting (CF) that consists of a model drastically decreasing its performance on previously learned tasks when it is sequentially trained on new tasks.This situation, known as task interference, occurs when a network modifies relevant weight values as it learns a new task.In this work, we propose two main strategies to face the problem of task interference in convolutional neural networks.First, we use a sparse coding technique to adaptively allocate model capacity to different tasks avoiding interference between them.Specifically, we use a strategy based on group sparse regularization to specialize groups of parameters to learn each task.Afterward, by adding binary masks, we can freeze these groups of parameters, using the rest of the network to learn new tasks.Second, we use a meta learning technique to foster knowledge transfer among tasks, encouraging weight reusability instead of overwriting.Specifically, we use an optimization strategy based on episodic training to foster learning weights that are expected to be useful to solve future tasks.Together, these two strategies help us to avoid interference by preserving compatibility with previous and future weight values.Using this approach, we achieve stateof-the-art results on popular benchmarks used to test techniques to avoid CF.In particular, we conduct an ablation study to identify the contribution of each component of the proposed method, demonstrating its ability to avoid retroactive interference with previous tasks and to promote knowledge transfer to future tasks.

This paper describes the submission of the IALab group of the Pontifical Catholic University of Chile to the Medical Domain Visual Question Answering (VQA-Med) task. Our participation was rather simple: we approached the problem as image classification. We took a DenseNet121 with its weights pre-trained in ImageNet and fine-tuned it with the VQA-Med 2020 dataset labels to predict the answer. Different answers were treated as different classes, and the questions were disregarded for simplicity since essentially they all ask for abnormalities. With this very simple approach we ranked 7th among 11 teams, with a test set accuracy of 0.236.

This article describes PUC Chile team’s participation in the Caption Prediction task of ImageCLEFmedical challenge 2021, which resulted in the team winning this task. We first show how a very simple approach based on statistical analysis of captions, without relying on images, results in a competitive baseline score. Then, we describe how to improve the performance of this preliminary submission by encoding the medical images with a ResNet CNN, pre-trained on ImageNet and later fine-tuned with the challenge dataset. Afterwards, we use this visual encoding as the input for a multi-label classification approach for caption prediction. We describe in detail our final approach, and we conclude by discussing some ideas for future work.

This article describes the participation and results of the PUC Chile team in the Turberculosis task in the context of ImageCLEFmedical challenge 2021. We were ranked 7th based on the kappa metric and 4th in terms of accuracy. We describe three approaches we tried in order to address the task. Our best approach used 2D images visually encoded with a DenseNet neural network, which representations were concatenated to finally output the classification with a softmax layer. We describe in detail this and other two approaches, and we conclude by discussing some ideas for future work.

This article describes PUC Chile team’s participation in the Concept Detection task of ImageCLEFmedical challenge 2021, which resulted in the team earning the fourth place. We made two submissions, the first one based on a naive approach which resulted in a F-1 score of 0.141, and an improved version which leveraged the Perceptual Similarity among images and obtained a final F-1 score of 0.360. We describe in detail our data analysis, our different approaches, and conclude by discussing some ideas for future work.