Models of second language (L2) word difficulty traditionally rely on corpus frequency and lexical indices derived from L1-English norms. Knowledge-based vocabulary lists provide a population-specific alternative, but testing thousands of words with sufficiently large samples is difficult to repeat across L1 groups. Crowdsourced yes/no datasets may offer a scalable alternative, although their value for estimating lexical difficulty cannot be assumed because yes/no judgments are less demanding than supervised meaning-based tests. We examined whether L1-specific yes/no familiarity predicts independently measured vocabulary difficulty beyond corpus- and L1-English-derived predictors. Familiarity estimates were derived separately from Arabic-L1 (n = 14,193) and Chinese-L1 (n = 38,633) respondents and used to predict group-specific Rasch-calibrated difficulty for 160 items administered to English-for-academic-purposes learners (N = 530). Hierarchical regression entered subtitle frequency first (H1), the remaining English-L1-based predictors second (H2), and L2-speaker-derived familiarity third (H3). H2 models accounted for 37.9% of Arabic item-difficulty variance and 42.1% of Chinese item-difficulty variance. In H3, L2-speaker-derived word familiarity remained significant in both groups, added ΔR2 = .143 for Arabic and ΔR2 = .050 for Chinese, and attenuated most English-based coefficients without eliminating all independent associations. Response-level binomial generalized linear mixed models with crossed random intercepts supported the familiarity effects. The findings support L1-specific yes/no familiarity as a practical, population-specific complement to English-based lexical difficulty predictors. To facilitate replication and further research, we provide Arabic- and Chinese-L1 Rasch familiarity estimates for 21,906 words, derived from the yes/no data collected by Brysbaert et al. (2021), as supplementary materials.

This study examined the construct validity of performance-based emotion recognition tests. We recruited 227 adults (30-60 years old) through Prolific to complete four emotion recognition tasks in addition to measures of self-reported empathy, crystallised intelligence, social confidence, loneliness/well-being, interest in people versus things, and reading enjoyment (all measured with at least two indicators). Consistent with previous research, performance-based emotion recognition tasks were positively correlated and formed a separate cluster. This cluster correlated with crystallised intelligence but not with self-reported emotion recognition skills, social-emotional functioning, interest in people vs. things, or reading pleasure. Overall, our findings suggest that performance-based emotion recognition tests primarily assess skills related to those measured by performance-based tests of cognitive intelligence, rather than skills related to the social-emotional functioning as experienced by the participants. This may partially explain the low correlation between subjective and performance-based measures of emotion recognition.

Calculating the reliability of experimental tasks can be surprisingly difficult using existing tools. Although R packages such as psych are robust, they often require data in wide format and assume carefully selected items that avoid floor and ceiling effects. To encourage the reporting of task reliability in experimental research, I have written an R function, ICC_participants_long, which uses the intraclass correlation coefficient (ICC) to measure the reliability of participant scores directly from data in long format. Applying this function revealed that the current split-half approach may underestimate the reliability of experimental tasks. Furthermore, the model-based approach makes it possible to generate Best Linear Unbiased Predictions (BLUPs) as estimates of participants’ scores, which provides an informative supplement to the raw means.

This study examined the concept of word usefulness by analyzing expected utility ratings for over 80,000 English words and multiword expressions. Participants used best-worst ratings to indicate how useful it is to know each word/expression. Our findings show a high level of agreement regarding the usefulness of words and expressions. Stimuli were rated as more useful if they were more frequent, widely known, learned early in life, and central to the semantic network. Concreteness had a substantial negative correlation, indicating that abstract words in general received higher utility scores than concrete words. Positive stimuli received slightly lower utility scores than negative stimuli. Expected utility was a good predictor of which words are known to speakers of English as a first and second language, but did not contribute to predicting response times to known words. These findings suggest that expected utility is a variable affecting which words are likely to be learned, but does not affect word processing times (much). The expected utility scores are freely available for research and education.

This article presents AI-generated estimates for five characteristics of German words: concreteness, valence, arousal, age of acquisition (AoA), and word familiarity. The estimates were generated using GPT-4o-mini, which was selected due to its good performance in previous studies. Validation studies were conducted comparing the AI-generated estimates with both human ratings and previously generated AI data to ensure their usefulness for research applications. The main results are as follows. The GPT estimates of word concreteness, valence, and arousal show a strong correlation with human ratings but are not better than the best available AI-generated estimates based on semantic vectors. The GPT estimates of AoA are good approximations of human ratings and outperform other available alternatives (except for human ratings), especially after the model was fine-tuned based on 2,000 human ratings. Fine-tuned AI-generated estimates of word familiarity have better predictive value than word frequency for word recognition in lexical decision tasks and vocabulary tests. Estimates for concreteness, valence, arousal, and AoA are available for 167,000 words, which are likely to be known to more than 90% of participants in typical adult studies. Word familiarity estimates are presented for 928,000 word forms. All data and codes, including newly collected human familiarity ratings for 11,000 words, are publicly available at https://osf.io/ghjd2/. The data may be freely used for research purposes, but not for commercial purposes.

This study introduces and validates GPT_FAM, an AI-generated resource of familiarity estimates for 935,000 Dutch words and 201,000 multiword expressions. Based on previous studies, we hypothesized that such estimates, particularly when fine-tuned using a few thousand human ratings, would offer a useful, scalable measure of verbal knowledge. The results confirmed the expectation, showing that fine-tuned GPT estimates correlate well with word prevalence, reflecting the likelihood of word recognition. Equally importantly, GPT_FAM estimates significantly predict response latencies in lexical decision tasks, emerging as the most robust predictor in a random forest analysis alongside word frequency and length. The measure may be especially useful for assessing the difficulty of morphologically complex items, such as inflected word forms and transparent compounds, where traditional frequency metrics tend to be ineffective. Both untuned and fine-tuned estimates are freely available for research and educational purposes.

This study evaluated the usefulness of AI-generated estimates of word familiarity for predicting word difficulty in Simplified Chinese, building on previous research in alphabetic languages. We found that familiarity estimates produced using large language models (LLMs) showed moderate-to-strong correlations with human familiarity ratings. These LLM estimates were the most effective predictors of both word naming and lexical decision times, surpassing traditional metrics such as word frequency and human familiarity ratings, while the latter still provided modest, non-overlapping variance. GPT-4o with English instructions produced superior results compared to the Chinese-centered models currently available. The results imply that LLM familiarity estimates are a valuable resource for Chinese psycholinguistics, supporting work across experimental design, modeling, and norming. We release familiarity estimates for 27,624 words for unrestricted research and educational use.

Word-level psycholinguistic norms are necessary to test theories of language processing. However, obtaining such human-based measures is not always feasible or straightforward. One promising approach is to augment human norming datasets by using large language models (LLMs) to predict these characteristics directly, a practice that is rapidly gaining popularity in psycholinguistics and cognitive science. However, the novelty of this approach (and the relative inscrutability of LLMs) necessitates the adoption of rigorous methodologies. We discuss the range of possible approaches, and clarify limitations that are not immediately apparent. In this work, we present a comprehensive methodology for estimating word characteristics with LLMs, enriched with practical advice and lessons learned from our own experience. Our approach covers both the direct use of base LLMs and the fine-tuning of models, an alternative that can yield substantial performance gains in certain scenarios. A major emphasis in the guide is the need to validate LLM-generated data, at least with a small set of a few hundred human “gold standard” norms, before using the LLM-generated norms. We also present a software framework that implements our methodology and supports both commercial and open-weight models. We illustrate the proposed approach with a case study on estimating word familiarity in English. Using base models, we achieved a Spearman correlation of 0.8 with human ratings, which increased to 0.9 when employing fine-tuned models. This methodology, framework, and set of best practices can serve as a reference for future research on leveraging LLMs for psycholinguistic and lexical studies.

Becoming a skilled reader requires that children accumulate extensive experience with text through independent reading. Research shows that greater text experience is associated with stronger reading skills, better comprehension, and improved spelling, and, consequently, higher reading motivation. Reliable objective measures of children’s reading experience are therefore essential; however, because such measures are typically highly sensitive to temporal and cultural contexts, none of the existing tests are suitable for capturing the reading experience of British children today. We address this gap by introducing a new Author Recognition Test (ART) and Title Recognition Test (TRT) designed specifically for primary school children in the United Kingdom and validated with a large cohort of British pupils. The battery also includes a new multiple-choice spelling test that can be easily administered online. We further demonstrate that single-word reading and sentence reading efficiency tests from the Rapid Online Assessment of Reading (ROAR) can be adapted for use with British children and provide valid measures of reading proficiency. Together, these tools offer a much-needed, freely available resource for both researchers and practitioners, enabling reliable measurement of children’s text experience and basic literacy skills. The test battery is openly available on https://osf.io/gmv72/.

This study compared the information yield of 5- and 7-point Likert scales in language research, motivated by theoretical arguments concerning the limits of working memory and response category confusion (i.e., that participants may not be able to attend to more than five response categories at once). An empirical investigation of subjective word frequency ratings confirmed that the 7-point scale did not produce estimates of frequency that were more highly correlated with objective frequency counts than those produced by the 5-point scale. However, this equivalence was not due to the hypothesized limits of human working memory capacity but rather to the equivalent precision achieved when estimating a regression line with five versus seven data points. These conclusions apply to rating scales that utilize the full variable range. In contexts where the response range utilizes only part of the Likert scale, the scale's effective precision may drop significantly, making more refined scales beneficial. Consequently, since the 7-point scale did not demonstrably increase participant effort or confusion, and since not all research questions involve the full Likert scale, researchers may prefer the 7-point scale for its versatility.

This article investigates the pervasive and often subtle dominance of English in academic scholarship. Drawing on existing literature and illustrative case studies, the analysis demonstrates and examines the deep-rooted prevalence of this phenomenon, particularly its compulsory nature for language researchers operating outside English-speaking contexts. The article concludes by proposing actionable solutions designed to foster greater inclusion of participants from diverse linguistic backgrounds.

Large Language Models (LLMs) exhibit a significant "embodiment gap", where their text-based representations fail to align with human sensorimotor experiences. This study systematically investigates whether and how task-specific fine-tuning can bridge this gap. Utilizing Representational Similarity Analysis (RSA) and dimension-specific correlation metrics, we demonstrate that the internal representations of LLMs can be steered toward more embodied, grounded patterns through fine-tuning. Furthermore, the results show that while sensorimotor improvements generalize robustly across languages and related sensory-motor dimensions, they are highly sensitive to the learning objective, failing to transfer across two disparate task formats.

Age of acquisition (AoA) is a widely used variable that estimates when a lexical item is first understood. Existing English AoA norms have been highly influential in psycholinguistics, education, language acquisition, speech-language pathology, and natural language processing, but have focused primarily on single words. Little information is availed for multi-word expressions (MWEs), despite their central role in language use, vocabulary acquisition, representation and processing. The current study contributes AoA estimates for 80,586 English MWEs using a large language model, GPT-4.1-mini, fine-tuned on newly collected crowdsourced human ratings. Ratings were obtained from 96 US-based native English speakers via Prolific, yielding 47,163 ratings for ~3,999 MWEs. After reliability screening, 3,667 BLUP-adjusted means were used for LLM fine-tuning and validation. Fine-tuning substantially improved alignment with hold-out human ratings. The standard GPT-4.1-mini output correlated with human estimates at r = .67, whereas the model fine-tuned on 3,000 items reached r = .85. A final model trained on all reliable crowdsourced estimates was estimated AoAs for the full MWE list. Results showed relationships with existing psycholinguistic variables aligned with those for single-word AoAs, including that earlier-acquired MWEs tended to be more familiar, useful, and frequent. The estimates exhibited predictive validity against test-based student vocabulary data and explained additional variance beyond frequency, utility, and familiarity. These findings indicate that fine-tuned LLMs can provide useful large-scale AoA estimates for MWEs when grounded in human ratings. The new resource is available via OSF (https://tinyurl.com/3e828fj8) and an interactive webpage has been developed for users: https://cgg-projects.github.io/MWEs/.

This paper revisits the age-of-acquisition (AoA) norms of Kuperman et al. (2012). Three studies were conducted. Study 1 reports a crowdsourcing 'megastudy' obtaining 790,024 estimates from participants with the age they could first read and write 11,074 early acquired words from Kuperman et al. (2012). The study aimed to differentiate between oral language receptive AoA and print-based AoA. The results correlate well with the original estimates, offering, as hypothesized, higher AoAs for reading/writing. These are released as supplements to the original norms. Study 2 explored the potential of large language models (LLMs), specifically GPT-4o, to replicate these crowdsourced AoA estimates. The findings indicated a strong correlation between AI-generated estimates and human judgments, showing the utility of AI in estimating AoA and developing norms for psycholinguistic and educational research in lieu of crowdsourcing. Study 3 leveraged AI to extend estimates to all well-known words in Kuperman et al. (2012) and the English Crowdsourcing Project (ECP). Study 3 also investigated a trained model fine-tuned on 2000 ratings from Kuperman et al. (2012). Fine-tuning increased alignment with human ratings, though comparisons with untrained models suggested that fine-tuning is not essential in English for obtaining useful AoA estimates. Both trained and untrained AI-generated norms correlated highly with human ratings and performed well in accounting for word processing times and accuracy in regressions. Uses and limitations of the AI estimates are discussed. All resources are made available in the Open Science Framework and can be used freely for research and education.

The evaluation of LLMs has so far focused primarily on how well they can perform different tasks such as reasoning, question-answering, paraphrasing, or translating. For most of these tasks, performance can be measured with objective metrics, such as the number of correct answers. However, other language features are not easily quantified. For example, arousal, concreteness, or gender associated with a given word, as well as the extent to which we experience words with senses and relate them to a specific sense. Those features have been studied for many years by psycholinguistics, conducting large-scale experiments with humans to produce ratings for thousands of words. This opens an opportunity to evaluate how well LLMs align with human ratings on these word features, taking advantage of existing studies that cover many different language features in a large number of words. In this paper, we evaluate the alignment of a representative group of LLMs with human ratings on two psycholinguistic datasets: the Glasgow and Lancaster norms. These datasets cover thirteen features over thousands of words. The results show that alignment is generally better in the Glasgow norms evaluated (arousal, valence, dominance, concreteness, imageability, familiarity, and gender) than on the Lancaster norms evaluated (introceptive, gustatory, olfactory, haptic, auditory, and visual). This suggests a potential limitation of current LLMs in aligning with human sensory associations for words, which may be due to their lack of embodied cognition present in humans and illustrates the usefulness of evaluating LLMs with psycholinguistic datasets.

A large portion of words in a language are formed by combining smaller meaningful units called morphemes (e.g., teach + -er → teacher). Understanding a language’s morphology is vital for skilled reading as it allows readers to interpret both familiar and unfamiliar words (e.g., tweeter). It is widely agreed that children rely on reading experience to acquire morpheme knowledge in English, and emerging research suggests that different aspects of this experience may impact affix learning in different ways. We contrasted three potential definitions of what constitutes readers’ affix experience using the morpheme interference paradigm with 120 adults. We found that skilled readers’ affix knowledge most closely aligns with a definition proposing that affix learning is primarily supported by experience with words in which affixes are identifiable without specialised linguistic knowledge. Due to the nature of morpheme presentation in English orthography, this excludes a significant number of genuinely complex words, while including affix-like patterns in non-meaningful contexts (e.g., -er in corner). This definition also posits that these morphological false alarms actively hinder learning. Our research represents a critical step towards a psychologically realistic theory of morpheme learning from text experience.

In this commentary, I express my concern that the special issue focuses too much on the added value of AI for psychology, while psychological research also has much to offer, such as the operationalization of variables based on theory, validation tools and the statistical evaluation of information generated by AI systems.