Quantization is widely used to reduce the inference cost of large language models, but its effect on reasoning models is not fully captured by final-answer accuracy or per-token latency. We show that low-bit post-training quantization can introduce a hidden test-time compute cost: quantized reasoning models often generate longer chains of thought even when they still answer correctly. Across mathematical reasoning, code generation, scientific question answering, and agentic tool-use benchmarks, we find that INT4/INT3 quantization can preserve accuracy but increase reasoning-token usage, offsetting the expected per-token speedup. To measure this effect, we introduce the CoT Token Inflation Ratio, which compares reasoning length between quantized and full-precision models averaged across all evaluation benchmarks. We further show that token inflation is accompanied by behavioral changes in the reasoning trace, including more intermediate steps and greater semantic repetition. These changes translate into measurable end-to-end real-world serving penalties. Finally, we evaluate mitigation strategies and find that prompting and decoding-time sampling offer inconsistent accuracy-length trade-offs, while quantization-aware training shows more promise in reducing both accuracy degradation and token inflation. Our results suggest that reasoning-token usage should be reported alongside accuracy when evaluating quantized reasoning models.

Federated learning (FL) has garnered considerable attention owing to its capability of accomplishing model training through the sharing local models without accessing training datasets. Nevertheless, it has been demonstrated that the shared models still possess sensitive information related to the training data. Moreover, there is a possibility that malicious aggregation servers can return manipulated global models. While the verification problem in FL has been explored in existing schemes, most of these schemes employ bilinear pairing operations and homomorphic hash computations dependent on the model's dimension, leading to substantial computational costs. Additionally, some schemes necessitate multiple parties to collectively manage one or more sets of confidential keys for privacy preservation and validation, which renders them vulnerable to collusion attacks between certain clients and servers. Consequently, we propose a privacy-preserving federated learning mechanism under a dual-server architecture. This mechanism adopts a coding matrix computation-based approach to ensure the privacy security of local models at the client side and achieves the aggregation of local models through collaborative efforts between two servers situated at the server side. To verify the correctness of the aggregated model, a Model Verification Code (MVC) mechanism is designed. By effectively combining the MVC mechanism with the coded matrix computation, there is no requirement for all clients to possess identical sets of confidential keys during the privacy preservation and verification process. Meanwhile, this ensures the fulfillment of security requirements under the malicious threat posed by the server. The computational overhead of this mechanism remains low since it avoids the application of complex cryptographic primitives. We perform extensive experiments on real datasets, and the experimental results further demonstrate the proposed scheme exhibits lightweight characteristics while ensuring the validity and usability of the model.

Accurately gauging the confidence level of Large Language Models' (LLMs) predictions is pivotal for their reliable application. However, LLMs are often uncalibrated inherently and elude conventional calibration techniques due to their proprietary nature and massive scale. In this work, we derive model confidence from the distribution of multiple randomly sampled generations, using three measures of consistency. We extensively evaluate eleven open and closed-source models on nine reasoning datasets. Results show that consistency-based calibration methods outperform existing post-hoc approaches in terms of calibration error. Meanwhile, we find that factors such as intermediate explanations, model scaling, and larger sample sizes enhance calibration, while instruction-tuning makes calibration more difficult. Moreover, confidence scores obtained from consistency can potentially enhance model performance. Finally, we offer guidance on choosing suitable consistency metrics for calibration, tailored to model characteristics such as the exposure to instruction-tuning and RLHF.

The privacy-protected algorithm (PPA) is pivotal in the realm of machine learning, especially for handling sensitive data types, such as medical and financial records. PPA enables two distinct operations: data publishing and data analysis, each capable of functioning independently. However, the field lacks a unified framework or an efficient algorithm to synergize these operations. This deficiency inspires our current research endeavor. In this paper, we introduce a novel dual-mode empirical risk minimization (D-ERM) model, specifically designed for integrated learning tasks. We also develop an alternating minimization differential privacy protection algorithm (AMDPPA) for implementing the D-ERM model. Our theoretical analysis confirms the differential privacy and accuracy of AMDPPA. We validate the algorithm’s efficacy through numerical experiments using real-world datasets, demonstrating its ability to effectively balance privacy with learning efficiency.

The issue of data privacy protection must be considered in distributed federated learning (FL) so as to ensure that sensitive information is not leaked. In this article, we propose a two-stage differential privacy (DP) framework for FL based on edge intelligence. Various levels of privacy preservation can be provided according to the degree of data sensitivity. In the first stage, the randomized response mechanism is used to perturb the original feature data by the user terminal for data desensitization, and the user can self-regulate the level of privacy preservation. In the second stage, noise is added to the local models by the edge server to further guarantee the privacy of the models. Finally, the model updates are aggregated in the cloud. In order to evaluate the performance of the proposed end-edge-cloud FL framework in terms of training accuracy and convergence, extensive experiments are conducted on a real electrocardiogram (ECG) signal dataset. Bi-directional long-short-term memory (BiLSTM) neural network is adopted to training classification model. The effect of different combinations of feature perturbation and noise addition on the model accuracy is analyzed depending on different privacy budgets and parameters. The experimental results demonstrate that the proposed privacy-preserving framework provides good accuracy and convergence while ensuring privacy.

We study a class of private learning problems in which the data is a join of private and public features. This is often the case in private personalization tasks such as recommendation or ad prediction, in which features related to individuals are sensitive, while features related to items (the movies or songs to be recommended, or the ads to be shown to users) are publicly available and do not require protection. A natural question is whether private algorithms can achieve higher utility in the presence of public features. We give a positive answer for multi-encoder models where one of the encoders operates on public features. We develop new algorithms that take advantage of this separation by only protecting certain sufficient statistics (instead of adding noise to the gradient). This method has a guaranteed utility improvement for linear regression, and importantly, achieves the state of the art on two standard private recommendation benchmarks, demonstrating the importance of methods that adapt to the private-public feature separation.

Pretrained Language Models (PLMs) have advanced Natural Language Processing (NLP) tasks significantly, but finetuning PLMs on low-resource datasets poses significant challenges such as instability and overfitting. Previous methods tackle these issues by finetuning a strategically chosen subnetwork on a downstream task, while keeping the remaining weights fixed to the pretrained weights. However, they rely on a suboptimal criteria for sub-network selection, leading to suboptimal solutions. To address these limitations, we propose a regularization method based on attention-guided weight mixup for finetuning PLMs. Our approach represents each network weight as a mixup of task-specific weight and pretrained weight, controlled by a learnable attention parameter, providing finer control over sub-network selection. Furthermore, we employ a bi-level optimization (BLO) based framework on two separate splits of the training dataset, improving generalization and combating overfitting. We validate the efficacy of our proposed method through extensive experiments, demonstrating its superiority over previous methods, particularly in the context of finetuning PLMs on low-resource datasets.

In this work, the federated learning mechanism is introduced into the deep learning of medical models in Internet of Things (IoT)-based healthcare system. Cryptographic primitives, including masks and homomorphic encryption, are applied for further protecting local models, so as to prevent the adversary from inferring private medical data by various attacks such as model reconstruction attack or model inversion attack, etc. The qualities of the datasets owned by different participants are considered as the main factor for measuring the contribution rate of the local model to the global model in each training epoch, instead of the size of datasets commonly used in deep learning. A dropout-tolerable scheme is proposed in which the process of federated learning would not be terminated if the number of online clients is not less than a preset threshold. Through the analysis of the security, it shows that the proposed scheme satisfies data privacy. Computation cost and communication cost are also analyzed theoretically. Finally, skin lesion classification using training images provided by the HAM10000 medical dataset is set as an example of healthcare applications. Experimental results show that compared with existing schemes, the proposed scheme obtained promising results while ensuring privacy preserving.

Most recommender systems rely on user interaction data for personalization. Usually, the recommendation quality improves with more data. In this work, we study the quality implications when limiting user interaction data for personalization purposes. We formalize this problem and provide algorithms for selecting a smaller subset of user interaction data. We propose a selection method that picks the subset of a user’s history items that maximizes the expected recommendation quality. We show on well-studied benchmarks that it is possible to achieve high-quality results with small subsets of less than 10 items per user.

We study the problem of multi-task learning under user-level differential privacy, in which n users contribute data to m tasks, each involving a subset of users. One important aspect of the problem, that can significantly impact quality, is the distribution skew among tasks. Certain tasks may have much fewer data samples than others, making them more susceptible to the noise added for privacy. It is natural to ask whether algorithms can adapt to this skew to improve the overall utility. We give a systematic analysis of the problem, by studying how to optimally allocate a user's privacy budget among tasks. We propose a generic algorithm, based on an adaptive reweighting of the empirical loss, and show that when there is task distribution skew, this gives a quantifiable improvement of excess empirical risk. Experimental studies on recommendation problems that exhibit a long tail of small tasks, demonstrate that our methods significantly improve utility, achieving the state of the art on two standard benchmarks.

The need to compactly and robustly represent item-attribute relations arises in many important tasks, such as faceted browsing and recommendation systems. A popular machine learning approach for this task denotes that an item has an attribute by a high dot-product between vectors for the item and attribute -- a representation that is not only dense, but also tends to correct noisy and incomplete data. While this method works well for queries retrieving items by a single attribute (such as \emph{movies that are comedies}), we find that vector embeddings do not so accurately support compositional queries (such as movies that are comedies and British but not romances). To address these set-theoretic compositions, this paper proposes to replace vectors with box embeddings, a region-based representation that can be thought of as learnable Venn diagrams. We introduce a new benchmark dataset for compositional queries, and present experiments and analysis providing insights into the behavior of both. We find that, while vector and box embeddings are equally suited to single attribute queries, for compositional queries box embeddings provide substantial advantages over vectors, particularly at the moderate and larger retrieval set sizes that are most useful for users' search and browsing.

We consider the problem of training private recommendation models with access to public item features. Training with Differential Privacy (DP) offers strong privacy guarantees, at the expense of loss in recommendation quality. We show that incorporating public item features during training can help mitigate this loss in quality. We propose a general approach based on collective matrix factorization (CMF), that works by simultaneously factorizing two matrices: the user feedback matrix (representing sensitive data) and an item feature matrix that encodes publicly available (non-sensitive) item information. The method is conceptually simple, easy to tune, and highly scalable. It can be applied to different types of public item data, including: (1) categorical item features; (2) item-item similarities learned from public sources; and (3) publicly available user feedback. Furthermore, these data modalities can be collectively utilized to fully leverage public data. Evaluating our method on a standard DP recommendation benchmark, we find that using public item features significantly narrows the quality gap between private models and their non-private counterparts. As privacy constraints become more stringent, models rely more heavily on public side features for recommendation. This results in a smooth transition from collaborative filtering to item-based contextual recommendations.

Matrix factorization learned by implicit alternating least squares (iALS) is a popular baseline in recommender system research publications. iALS is known to be one of the most computationally efficient and scalable collaborative filtering methods. However, recent studies suggest that its prediction quality is not competitive with the current state of the art, in particular autoencoders and other item-based collaborative filtering methods. In this work, we revisit four well-studied benchmarks where iALS was reported to perform poorly and show that with proper tuning, iALS is highly competitive and outperforms any method on at least half of the comparisons. We hope that these high quality results together with iALS’s known scalability spark new interest in applying and further improving this decade old technique.

The application of vehicle ad-hoc networks has brought huge potential to the development of intelligent transportation systems. By traffic event reporting, traffic efficiency can be significantly improved. However, people usually lack the motivation for reporting traffic events or driving violations, because once the identity of the reporter is leaked, he or she may be tracked by malicious attackers or be retaliated by violators. However, if the reporter is anonymous, it is difficult to guarantee the accuracy and authenticity of the reports. In order to solve this contradiction, a mechanism that combines certificateless message authentication and blockchain incentives is proposed in this paper. A certificateless message signature algorithm is designed to provide anonymity and non-repudiation for traffic-related message reporters. Simultaneously, an adaptive t-threshold multi-signature mechanism is introduced in our scheme. Similar to crowdsourcing, it requires multiple participants to witness the authenticity of the message together. Aggregation signature verification improves message authentication efficiency. A trust currency is introduced to motivate reporters’ enthusiasm for participation in intelligent transportation, and it is also taken as a report dissemination network. The simulation results show that our scheme is secure and effective in the intelligent transportation system.

We present a multi-level geocoding model (MLG) that learns to associate texts to geographic coordinates. The Earth’s surface is represented using space-filling curves that decompose the sphere into a hierarchical grid. MLG balances classification granularity and accuracy by combining losses across multiple levels and jointly predicting cells at different levels simultaneously. It obtains large gains without any gazetteer metadata, demonstrating that it can effectively learn the connection between text spans and coordinates—and thus makes it a gazetteer-free geocoder. Furthermore, MLG obtains state-of-the-art results for toponym resolution on three English datasets without any dataset-specific tuning.

We present ALX, an open-source library for distributed matrix factorization using Alternating Least Squares, written in JAX. Our design allows for efficient use of the TPU architecture and scales well to matrix factorization problems of O(B) rows/columns by scaling the number of available TPU cores. In order to spur future research on large scale matrix factorization methods and to illustrate the scalability properties of our own implementation, we also built a real world web link prediction dataset called WebGraph. This dataset can be easily modeled as a matrix factorization problem. We created several variants of this dataset based on locality and sparsity properties of sub-graphs. The largest variant of WebGraph has around 365M nodes and training a single epoch finishes in about 20 minutes with 256 TPU cores. We include speed and performance numbers of ALX on all variants of WebGraph. Both the framework code and the dataset is open-sourced.

Being able to efficiently and accurately select the top-$k$ elements with differential privacy is an integral component of various private data analysis tasks. In this paper, we present the oneshot Laplace mechanism, which generalizes the well-known Report Noisy Max mechanism to reporting noisy top-$k$ elements. We show that the oneshot Laplace mechanism with a noise level of $\widetilde{O}(\sqrt{k}/\eps)$ is approximately differentially private. Compared to the previous peeling approach of running Report Noisy Max $k$ times, the oneshot Laplace mechanism only adds noises and computes the top $k$ elements once, hence much more efficient for large $k$. In addition, our proof of privacy relies on a novel coupling technique that bypasses the use of composition theorems. Finally, we present a novel application of efficient top-$k$ selection in the classical problem of ranking from pairwise comparisons.

Privacy-preserving empirical risk minimization model is crucial for the increasingly frequent setting of analyzing personal data, such as medical records, financial records, etc. Due to its advantage of a rigorous mathematical definition, differential privacy has been widely used in privacy protection and has received much attention in recent years of privacy protection. With the advantages of iterative algorithms in solving a variety of problems, like empirical risk minimization, there have been various works in the literature that target differentially private iteration algorithms, especially the adaptive iterative algorithm. However, the solution of the final model parameters is imprecise because of the vast privacy budget spending on the step size search. In this paper, we first proposed a novel adaptive differential privacy algorithm that does not require the privacy budget for step size determination. Then, through the theoretical analyses, we prove that our proposed algorithm satisfies differential privacy, and their solutions achieve sufficient accuracy by infinite steps. Furthermore, numerical analysis is performed based on real-world databases. The results indicate that our proposed algorithm outperforms existing algorithms for model fitting in terms of accuracy.

We study the problem of differentially private (DP) matrix completion under user-level privacy. We design a joint differentially private variant of the popular Alternating-Least-Squares (ALS) method that achieves: i) (nearly) optimal sample complexity for matrix completion (in terms of number of items, users), and ii) the best known privacy/utility trade-off both theoretically, as well as on benchmark data sets. In particular, we provide the first global convergence analysis of ALS with noise introduced to ensure DP, and show that, in comparison to the best known alternative (the Private Frank-Wolfe algorithm by Jain et al. (2018)), our error bounds scale significantly better with respect to the number of items and users, which is critical in practical problems. Extensive validation on standard benchmarks demonstrate that the algorithm, in combination with carefully designed sampling procedures, is significantly more accurate than existing techniques, thus promising to be the first practical DP embedding model.

Differential privacy provides a rigorous framework for privacy-preserving data analysis. This paper proposes the first differentially private procedure for controlling the false discovery rate (FDR) in multiple hypothesis testing. Inspired by the Benjamini-Hochberg procedure (BHq), our approach is to first repeatedly add noise to the logarithms of the p-values to ensure differential privacy and to select an approximately smallest p-value serving as a promising candidate at each iteration; the selected p-values are further supplied to the BHq and our private procedure releases only the rejected ones. Moreover, we develop a new technique that is based on a backward submartingale for proving FDR control of a broad class of multiple testing procedures, including our private procedure, and both the BHq step- up and step-down procedures. As a novel aspect, the proof works for arbitrary dependence between the true null and false null test statistics, while FDR control is maintained up to a small multiplicative factor.