The problem of improving the quality of intelligent search in a corpus of educational institution documents, including curricula, course syllabus, and regulatory documents, has been solved. Classic architectures of the retrieval-augmented generation (RAG) approach, based on a single plaintext search module, demonstrate low accuracy when used with documents containing tables, logical relationships between entities, and strict regulatory document wording. An adaptive RAG architecture is proposed, consisting of four layers, each taking into account the specifics of the data storage in such documents. The results indicate that considering document structure and adaptive query routing significantly improve the actual accuracy of responses. The proposed architecture is usable in the design of intelligent assistants for administrative and educational services at higher education institutions.
A fuzzy controller for calculating sliding window parameters is proposed to prepare training data for large language models. Traditional approaches set the “window step” and “context length” as fixed constants across the entire text without considering linguistic characteristics of individual fragments, such as rich scientific text and monotonous repeating text. The proposed method utilizes two automatically extracted features: lexical diversity and average BPE token length. Operating on the Mamdani inference algorithm with a nine-rule base and center-of-gravity defuzzification, the fuzzy controller adaptively calculates the “window step” and “context length” parameters for each fragment. The proposed approach offers a cognitive interpretation by mimicking human reading mechanics, where readers naturally apply adaptive attention; for example, complex fragments are processed more attentively using smaller incremental steps.
Large language models (LLMs) are increasingly being used for code generation. However, both the generated code and the underlying LLM-based systems demand rigorous security evaluation. A common approach to enhancing code generation quality involves multi-agent systems composed of multiple models. This paper evaluates the performance of GPT-OSS 20B, GPT-OSS 120B, and Qwen3-Coder 480B models in single-agent and multi-agent configurations, using two code security benchmarks, SecurityEval and CyberSecEval. The key contribution is SafeAICoder, an extensible and scalable framework for testing LLMs enabling distributed server-side generation of multi-module programs and tests, independent of client-side code.
Task-specific models have long dominated natural language processing tasks. However, general-purpose large language models (LLMs) have recently begun to successfully compete with highly specialized solutions across various NLP domains. This paper examines the applicability of LLMs to the problem of estimating the semantic proximity of word meanings in a pair of usages, known as Word-in-Context (WiC). Drawing on the multilingual CoMeDi benchmark, we propose novel approaches to building automated WiC systems based on LLMs. Five different configurations of LLM-based WiC systems were systematically compared in terms of their quality and computational cost. In particular, we propose a configuration where LLM predictions are adjusted using a training set, without the need to fine-tune the LLM itself. Results on test sets across seven languages show that our proposed approaches enable LLMs to outperform all existing specialized systems, establishing a new state-of-the-art (SOTA) on the CoMeDi benchmark. However, this higher quality comes at the cost of a significant rise in computing costs: LLM-based systems require several orders of magnitude more computations compared to compact, specialized models (such as XL-DURel). This work is one step toward understanding the trade-off between accuracy and resource efficiency when using modern LLMs in lexical semantics tasks.
This article addresses the problem of quantifying the confidence of large language models (LLMs) in the automatic semantic classification of arguments with emotive predicates. Using Russian-language social media posts, we analyze verbs of fear (to scare, to be afraid, etc.) and emotional attitude (to like, to love) with the semantic roles of experiencer, causator, and object. The study aims to compare self-assessed confidence of LLM Claude Sonnet 4.5 with expert assessments of the model’s reasoning texts in argument classification in the healthcare domain. The experiment utilized a stratified set of 300 examples using chain-of-thoughts reasoning in Russian and a four-step confidence scale. The results showed a moderate Spearman’s correlation between the expert and model assessments. A statistically significant relationship was found only between self-assessment of the model and the actual classification accuracy, while expert assessment of the linguistic characteristics of reasoning was unrelated to accuracy. It was concluded that explicit LLM reasoning is not directly related to its self-assessment of confidence and is separate from the decision-making process; reasoning may be an important functional part of the user interface, but not that of the research.
