Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller model to draft future tokens, which are then verified by the target LLM. This preserves generation quality by accepting only aligned tokens. However, individual drafters, often trained for specific tasks or domains, exhibit limited effectiveness across diverse applications. To address this, we introduce MetaSD, a unified framework that integrates multiple drafters into the SD process. MetaSD dynamically allocates computational resources to heterogeneous drafters by leveraging alignment feedback and framing drafter selection as a multi-armed bandit problem. Extensive experiments show MetaSD consistently outperforms single-drafter approaches.

Convolutional neural networks (CNNs) achieve strong performance in vision, but their convolutional operators and feature hierarchies can impose non-trivial compute and parameter overhead in lightweight settings. Meanwhile, fully connected (MLP-based) vision models often sacrifice key CNN inductive biases such as locality and weight sharing. In this paper, we present MP-MLP (Micro-Patch Multi-Layer Perceptron), a convolution-free lightweight architecture that recovers CNN-like behavior using only fully connected layers. MP-MLP partitions an input image into non-overlapping micro-patches and applies a single weight-shared MLP block to every patch, acting as a pseudo-convolutional filter without any convolution operations. Patch-wise features are concatenated and fed into a shallow classifier MLP for end-to-end recognition. We evaluate MP-MLP on MNIST, Fashion-MNIST, and SVHN, covering increasing task complexity from clean grayscale digits to RGB street-view digits. With substantially fewer parameters, MP-MLP achieves competitive accuracy on MNIST and FashionMNIST, and slightly outperforms a lightweight CNN baseline on SVHN, demonstrating that carefully designed weight-shared MLPs can be a compelling convolution-free alternative for structured lightweight vision tasks.

WiFi Channel State Information (CSI)-based sensing is a pivotal technology for device-free ambient intelligence. However, its practical adoption has been hindered by a heavy reliance on specialized hardware and computationally prohibitive models. This paper addresses these challenges by introducing CSI-Chain, a complete, open-source, and end-to-end framework designed for commodity ESP32 microcontrollers. CSI-Chain integrates a robust real-time preprocessing pipeline with high-performance inference to achieve state-of-the-art results in unified Human Activity Recognition (HAR) and indoor localization. Our contributions are fourfold: (1) A Complete & Reproducible Pipeline: We present the first fully integrated CSI sensing framework built exclusively on low-cost commodity hardware, fundamentally lowering the barrier to entry for practical deployment. (2) LiteCSINet: We present an efficient, purpose-built CNN architecture that achieves competitive performance with larger transformer-based models while using only one-third of the parameters, demonstrating that a well-designed lightweight model, when paired with robust preprocessing, can match heavyweight approaches. (3) Unified Multi-Task Learning: The framework effectively unifies HAR and zone-based localization within a single model, achieving high accuracies of 95.8% and 98.5%, respectively. (4) Cross-Environment Validation: We rigorously evaluate the system's robustness through cross-environment testing, proving its generalization capabilities a critical yet often overlooked requirement for real-world applications. CSI-Chain provides a viable blueprint for scalable and privacy-preserving smart environments, proving that high-performance sensing is achievable without expensive specialized hardware.

Low-Rank Adaptation (LoRA) has become a de facto standard for parameter-efficient fine-tuning (PEFT), yet its performance is highly sensitive to initialization due to the information bottleneck imposed by low-rank decomposition. Existing approaches attempt to construct high-quality LoRA initializations by exploiting principal components of pretrained weights, activations, or gradients. However, these methods do not directly account for the training dynamics of the full-rank model. In this paper, we propose Training-aware Low-Rank Adaptation Initialization (TaRA), a method that initializes LoRA such that the gradients induced by the low-rank factors closely approximate the gradient of the corresponding full-rank weight matrix. Derived from a mathematical formulation, TaRA improves gradient fidelity at the start of training while introducing negligible computational overhead. Across diverse and challenging fine-tuning tasks, TaRA consistently outperforms prior state-of-the-art methods, establishing a simple, robust, and scalable solution for effective LoRA initialization.

Large reasoning models (LRMs) achieve strong performance by externalizing explicit reasoning traces before producing the answer, yet suffer from overthinking challenge that allocates uniformly heavy computation to queries of varying difficulty. While proprietary models mitigate this via opaque routing, open-source LRMs still lack an efficient mechanism to internalize adaptive reasoning due to both expensive training cost and limited disclosure of training recipes. In response, we introduce RPO (Root-token Policy Optimization), a framework that enables LRMs to self-determine when to reason by training only the initial root token (e.g., whether to invoke the think tag) via group relative reward and group-wise advantages. By focusing on this pivotal branching point, RPO drastically reduces training overhead and VRAM usage. Across multiple model families and scales, RPO learns difficulty-aware adaptive thinking at just 2% of the training compute of prior adaptive-reasoning methods.

Bargaining is often regarded as a logical arena rather than an art or a matter of intuition, yet Large Language Models (LLMs) still struggle to navigate it due to limited strategic depth and difficulty adapting to complex human factors. Current benchmarks rarely capture this limitation. To bridge this gap, we present a utility feedback centric framework. Our contributions are: (i) AgoraBench, a new benchmark spanning nine challenging settings (e.g., deception, monopoly) that supports diverse strategy modeling; (ii) human-aligned, economically grounded metrics derived from utility theory. This is operationalized via agent utility, negotiation power, and acquisition ratio that implicitly measure how well the negotiation aligns with human preference and (iii) a human preference grounded dataset with learning pipeline that strengthens LLMs’ bargaining ability through both prompting and finetuning. Empirical results indicate that baseline LLM strategies often diverge from human preferences, while our mechanism substantially improves negotiation performance, yielding deeper strategic behavior and stronger opponent awareness.

Self-Taught Reasoners (STaR), synonymously known as Rejection sampling Fine-Tuning (RFT), is an integral part of the training pipeline of self-improving reasoning Language Models (LMs). The self-improving mechanism often employs random observation (data) sampling. However, this results in trained observation imbalance; inefficiently over-training on solved examples while under-training on challenging ones. In response, we introduce Adaptive STaR (AdaSTaR), a novel algorithm that rectifies this by integrating two adaptive sampling principles: (1) Adaptive Sampling for Diversity: promoting balanced training across observations, and (2) Adaptive Sampling for Curriculum: dynamically adjusting data difficulty to match the model's evolving strength. Across six benchmarks, AdaSTaR achieves best test accuracy in all instances (6/6) and reduces training FLOPs by an average of 58.6\% against an extensive list of baselines. These improvements in performance and efficiency generalize to different pre-trained LMs and larger models, paving the way for more efficient and effective self-improving LMs.

The limited reasoning capabilities of small language models (SLMs) cast doubt on their suitability for tasks demanding deep, multi-step logical deduction. This paper introduces a framework called Small Reasons, Large Hints (SMART), which selectively augments SLM reasoning with targeted guidance from large language models (LLMs). Inspired by the concept of cognitive scaffolding, SMART employs a score-based evaluation to identify uncertain reasoning steps and injects corrective LLM-generated reasoning only when necessary. By framing structured reasoning as an optimal policy search, our approach steers the reasoning trajectory toward correct solutions without exhaustive sampling. Our experiments on mathematical reasoning datasets demonstrate that targeted external scaffolding significantly improves performance, paving the way for collaborative use of both SLM and LLM to tackle complex reasoning tasks that are currently unsolvable by SLMs alone.

Generating high-quality charts with Large Language Models (LLMs) presents significant challenges due to limited data and the high cost of scaling through human curation. 'instruction' data, code. triplets are scarce and expensive to manually curate as their creation demands technical expertise. To address this scalability challenge, we introduce a reference-free automatic feedback generator, which eliminates the need for costly human intervention. Our novel framework, C-2, consists of (1) an automatic feedback provider (CHARTAF) and (2) a diverse, reference-free dataset (CHARTUIE-8K). The results are compelling: in our first experiment, 74% of respondents strongly preferred, and 10% preferred, the results after feedback. The second post-feedback experiment demonstrates that CHARTAF outperform nine baselines. Moreover, CHARTUIE-8K significantly improves data diversity by increasing queries, datasets, and chart types by 5982%, 1936%, and 91%, respectively, over benchmarks. Finally, a study of LLM users revealed that 94% of participants preferred CHARTUIE-8K's queries, with 93% deeming them aligned with real-world use cases. Core contributions are available as open-source at chartsquared.github.io, with ample qualitative examples.

The generation of 3D digital humans has traditionally relied on multi-view imaging systems and large-scale datasets, posing challenges in cost, accessibility, and real-time applicability. To overcome these limitations, this study presents an efficient pipeline that constructs high-fidelity 3D digital humans from a single frontal image. By leveraging generative AI, the system synthesizes additional views and generates UV maps compatible with the SMPL-X model, ensuring anatomically accurate and photorealistic reconstructions. The generated 3D models are imported into Unity 3D, where they are rigged for real-time motion synchronization using BlazePose-based lightweight pose estimation. To further enhance motion realism, custom algorithms—including ground detection and rotation smoothing—are applied, improving movement stability and fluidity. The system was rigorously evaluated through both quantitative and qualitative analyses. Results show an average generation time of 211.1 s, segmentation accuracy of 92.1%, and real-time rendering at 64.4 FPS. In qualitative assessments, expert reviewers rated the system using the SUS usability framework and heuristic evaluation, confirming its usability and effectiveness. This method eliminates the need for multi-view cameras or depth sensors, significantly reducing the barrier to entry for real-time 3D avatar creation and interactive AI-driven applications. It has broad applications in virtual reality (VR), gaming, digital content creation, AI-driven simulation, digital twins, and telepresence systems. By introducing a scalable and accessible 3D modeling pipeline, this research lays the groundwork for future advancements in immersive and interactive environments.

This paper introduces a lane detection module optimized for embedded systems, addressing the significant challenge of adapting to different road conditions that arise from the over-reliance on datasets from a single nation, primarily China. Traditional lane detection algorithms are often hindered by their dependency on extensive and diverse data that may not be available in all regions. By developing and utilizing a tailored Korean dataset, we effectively overcome the domain adaptation problem, ensuring that our model performs reliably under the unique road conditions found in Korea. Our approach not only utilizes depth images derived from camera-captured data, enhancing the training process without the need for multiple sensor types but also significantly reduces the resource demands typical of conventional sensor fusion techniques. Consequently, we present a highly efficient, Korean-specific lane detection model that demonstrates robust performance and is particularly suited for implementation within embedded systems, offering a scalable and practical solution in the realm of autonomous driving technologies.

We introduce the Block Transformer which adopts hierarchical global-to-local modeling to autoregressive transformers to mitigate the inference bottlenecks associated with self-attention. Self-attention requires the key-value (KV) cache of all previous sequences to be retrieved from memory at every decoding step to retrieve context information, leading to two primary bottlenecks during batch inference. First, there is a significant delay in obtaining the first token, as the information of the entire prompt must first be processed to prefill the KV cache. Second, computation of subsequent tokens is bottlenecked by the high memory I/O demand of fetching the entire KV cache, which grows linearly with sequence length, incurring quadratic memory reads overall. We design the Block Transformer to strategically mitigate these costs, by incorporating coarsity and locality into an integrated global-to-local architecture. At the lower layers, we aggregate tokens into fixed size blocks to apply attention across the entire sequence at coarse-grained detail, to capture the global context while minimizing KV cache overhead. At upper layers, we apply attention within each block to decode individual tokens, to model fine-grained details with a lightweight local KV cache. We pretrain vanilla and Block Transformers from scratch and demonstrate that Block Transformers reach 10--20x inference throughput compared to vanilla transformers with equivalent perplexity and zero-shot task performance.

Despite the remarkable strides made by autoregressive language models, their potential is often hampered by the slow inference speeds inherent in sequential token generation. Blockwise parallel decoding (BPD) was proposed by Stern et al. (2018) as a way to improve inference speed of language models. In this paper, we make two contributions to understanding and improving BPD drafts. We first offer an analysis of the token distributions produced by the BPD prediction heads. Secondly, we use this analysis to inform algorithms to improve BPD inference speed by refining the BPD drafts using small n-gram or neural language models. We empirically show that these refined BPD drafts yield a higher average verified prefix length across tasks.

Recent advancements in WiFi Channel State Information (CSI) sensing have paved the way for robust activity recognition and human identification; however, these methods often struggle with the inherent uncertainty due to environmental noise and the variability in human activity patterns. To address these challenges, we introduce an introspective deep metric learning (IDML) framework adapted for WiFi CSI Sensing. By leveraging an uncertainty-aware embedding, our approach not only captures the semantic features of human activities but also their associated uncertainties. This dual embedding system enables our model to better distinguish between different activities and individuals, even under noisy or ambiguous conditions. We propose a novel introspective similarity metric that considers both semantic differences and ambiguities, allowing for more accurate and robust classification and identification. Our method was evaluated using several standard datasets for activity recognition and human identification, where it demonstrated superior performance over traditional methods, particularly in handling complex scenarios with high ambiguity and noise. This work not only enhances the capabilities of WiFi CSI Sensing but also sets a new benchmark for uncertainty modeling in activity and identity recognition systems.

Federated Learning (FL) has emerged as a promising paradigm in which multiple clients collaboratively train a shared global model while preserving data privacy. To create a robust and practicable FL framework, it is crucial to extend its ability to generalize well to unseen domains - a problem referred to as federated Domain Generalization (FDG), being still under-explored. We propose an innovative federated algorithm, termed hFedF for hypernetwork-based Federated Fusion, designed to bridge the performance gap between generalization and personalization, capable of addressing various degrees of domain shift. Essentially, the hypernetwork supports a non-linear fusion of client models enabling a comprehensive understanding of the underlying data distribution. We encompass an extensive discussion and provide novel insights into the tradeoff between personalization and generalization in FL. The proposed algorithm outperforms strong benchmarks on three widely-used data sets for DG in an exceeding number of cases.

Label noise in federated learning (FL) has garnered increasing attention due to the decentralized nature of FL, where data is collected from multiple clients with potentially different levels of label noise. This study introduces two pivotal contributions to this domain. First, we anatomize the memorization phenomenon in FL into server-side and client-side components, marking the first investigation into how these distinct forms of memorization impact learning. Second, to mitigate the memorization in FL, we present the Federated Label-mixture Regularization (FLR) strategy, a straightforward yet effective approach that employs regularization through pseudo labels generated by merging local and global model predictions. This method not only improves the accuracy of the global model in both i.i.d. and non-i.i.d. settings but also effectively counters the memorization of noisy labels. We empirically find that FLR aligns with and advances existing FL and noisy label mitigation methods over multiple datasets under various levels of data heterogeneity and label noise.

Autoregressive language models have achieved remarkable advancements, yet their potential is often limited by the slow inference speeds associated with sequential token generation. Blockwise parallel decoding (BPD) was proposed by Stern et al. [42] as a method to improve inference speed of language models by simultaneously predicting multiple future tokens, termed block drafts, which are subsequently verified by the autoregressive model. This paper advances the understanding and improvement of block drafts in two ways. First, we analyze token distributions generated across multiple prediction heads. Second, leveraging these insights, we propose algorithms to improve BPD inference speed by refining the block drafts using task-independent \ngram and neural language models as lightweight rescorers. Experiments demonstrate that by refining block drafts of open-sourced Vicuna and Medusa LLMs, the mean accepted token length are increased by 5-25% relative. This results in over a 3x speedup in wall clock time compared to standard autoregressive decoding in open-source 7B and 13B LLMs.

Federated Learning (FL) faces significant challenges with domain shifts in heterogeneous data, degrading performance. Traditional domain generalization aims to learn domain-invariant features, but the federated nature of model averaging often limits this due to its linear aggregation of local learning. To address this, we propose a robust framework, coined as hypernetwork-based Federated Fusion (hFedF), using hypernetworks for non-linear aggregation, facilitating generalization to unseen domains. Our method employs client-specific embeddings and gradient alignment techniques to manage domain generalization effectively. Evaluated in both zero-shot and few-shot settings, hFedF demonstrates superior performance in handling domain shifts. Comprehensive comparisons on PACS, Office-Home, and VLCS datasets show that hFedF consistently achieves the highest in-domain and out-of-domain accuracy with reliable predictions. Our study contributes significantly to the under-explored field of Federated Domain Generalization (FDG), setting a new benchmark for performance in this area.

While instruction-tuned language models have demonstrated impressive zero-shot generalization, these models often struggle to generate accurate responses when faced with instructions that fall outside their training set. This paper presents Instructive Decoding (ID), a simple yet effective approach that augments the efficacy of instruction-tuned models. Specifically, ID adjusts the logits for next-token prediction in a contrastive manner, utilizing predictions generated from a manipulated version of the original instruction, referred to as a noisy instruction. This noisy instruction aims to elicit responses that could diverge from the intended instruction yet remain plausible. We conduct experiments across a spectrum of such noisy instructions, ranging from those that insert semantic noise via random words to others like 'opposite' that elicit the deviated responses. Our approach achieves considerable performance gains across various instruction-tuned models and tasks without necessitating any additional parameter updates. Notably, utilizing 'opposite' as the noisy instruction in ID, which shows the maximum divergence from the original instruction, consistently produces the most significant performance gains across multiple models and tasks.

There has been considerable research on reconstructing 3D shapes from single-view images; however, preserving the detailed information of the input image remains difficult. In this paper, we propose the application of a gradient map to train a network, aimed at improving the visual quality of fine-grained details such as the thin and tiny components of generated shapes. Each gradient map was created from the original voxel data, and each value represented the amount of information per volume. Here, the gradient map was defined by several methods that mathematically quantify and represent the detailed structure of an object. By applying this map to the loss function in training, we could induce the network to intensively train partial details, such as thin and narrow parts. We demonstrated that the detailed information was well-recovered when a weight that is proportional to the gradient value was applied to the loss. Furthermore, it is expected that our method will contribute to the development of 3D technologies related to the construction of virtual space for simulation and new customer experience.