Large language models learn word meanings through distributional semantics (i.e., patterns in text), and distribution does not always capture information about taxonomy and function, which are essential to human representations of word meaning. However, sound and spelling often conveys such information, and language models decompose many words into subword tokens, which may identify category markers. For example, GPT-5 segments fluorine and bromine into fluor + ine and brom + ine, which share the category marker ine. We provide evidence that, across 18 languages, shared tokens predict greater semantic priming effects in humans (e.g., people recognize bromine faster after reading fluorine), beyond what distributional similarity explains. Furthermore, across seven languages, shared tokens boost semantic priming beyond what shared morphemes explain. This suggests that subword tokens could help language models arrive at human-like representations of word meanings. However, we also provide evidence that the trend towards larger vocabularies of tokens obscures category markers, and even when language models have access to sub-word tokens, they underestimate the semantic relatedness of words that share tokens.
Abstract The high memory demands of generative language models have drawn attention to quantization, which reduces memory usage by mapping model weights to lower-precision integers. However, recent empirical studies show that, while efficient, quantization can increase the likelihood of generating biased outputs and degrade performance on fairness benchmarks. In this work, we draw new links between quantization and model fairness by adding explicit group-fairness constraints to the quantization objective and introduce Fair-GPTQ, the first quantization method explicitly designed to reduce unfairness in large language models. The added constraints guide the learning of the rounding operation toward less-biased text generation for protected groups. Specifically, we focus on stereotype generation involving occupational bias and discriminatory language spanning gender, race, and religion. Fair-GPTQ has minimal impact on performance, preserving at least 90% of baseline accuracy on zero-shot benchmarks, reduces unfairness relative to a half-precision model, and retains the memory and speed benefits of 4-bit quantization.
Abstract Cross-lingual adaptation remains a significant challenge for Large Language Models (LLMs) and Vision-Language Models (VLMs), particularly for low-resource languages. While the synthetic generation of code-switched text is an effective cross-lingual adaptation strategy, stochastic word replacement often degrades semantic integrity. We propose Semantic Code-Switching (SCS), a framework ensuring high-fidelity cross-lingual augmentation. We first train a Semantic Alignment Scorer to predict contextual similarity between matrix and code-switched sentences, then integrate this scorer into training pipelines via temperature-scaled sampling. Across diverse tasks, SCS consistently outperforms baselines like CoSDA-ML. Our method is especially robust for typologically distant languages, offering a scalable solution for multilingual adaptation without requiring parallel corpora.
Abstract Post-training can turn the same pretrained backbone into an instruction follower, a domain specialist, or a reasoning-distilled model. We ask whether these variants leave measurable internal traces and whether those traces improve inference-time decisions. We represent generated-token hidden states as layer-wise trajectories and summarize their length-residualized correctness associations in a metric-by-layer geometric signature. We evaluate observability, candidate-selection utility, and intervention efficacy separately. In retained retrospective artifacts collected under a fixed prompt, decoder, and problem slice, matched-width Qwen2.5-derived variants exhibit different signature matrices. Retrospective bootstrap, budget-sensitivity, and scale audits reveal recurring contrasts but protocol-sensitive nearest-neighbor topology; a separately preregistered fresh confirmation chain did not qualify because of its truncation gate. The signatures are therefore descriptive, protocol-bounded diagnostics, not universal fingerprints. Stricter controls change the downstream conclusions. On shared K=8 candidate pools, GeoVote (geometric voting) does not improve selection over majority, length, and likelihood controls. In the locked Qwen-Instruct intervention comparison, every source-derived, target-calibrated, conventional, sparse, sign-reversed, and matched-random row has a 95% paired confidence interval that includes zero relative to no steering. A separate corrected R1-Distill short-envelope rerun is also null but remains cap-saturated. Source transfer likewise fails to improve MATH-500, SVAMP, or long-context evaluations. Among six pre-declared label budgets, one five-label cell reaches +11 pp (Holm-adjusted p=0.0443), but the non-monotone curve does not exhibit a dose–response pattern and is not presented as a label-efficiency or transfer effect. We conclude that trajectory geometry provides a protocol-bounded, auditable measurement of post-training differences; selection utility and transferable control require separate evidence.
This study explores the capacity of Large Language Models (LLMs) to perform tasks requiring Theory of Mind (ToM), a critical component of pragmatic language understanding. Although previous work suggests that LLMs may exhibit emergent ToM abilities, this research examines whether such capabilities genuinely involve reasoning about beliefs or merely reflect the reliance on shallow statistical cues. Through a series of controlled experiments featuring indirect speech acts and verbal irony, we assess how belief contexts influence LLM interpretations. The results reveal that, although LLMs occasionally succeed in decoding communicative intentions, their performance is not attributable to human-like ToM reasoning. This work underscores the limitations of LLMs in simulating humanlike ToM and offers insight into their interpretive biases, contributing to a deeper understanding of their linguistic capabilities.(1)
