This review paper provides a comprehensive survey of Large Language Models (LLMs) applied to Document-Based Question Answering (DBQA). The rapid advancements in LLMs have transformed DBQA systems by enhancing their capabilities in complex reasoning, multi-hop question answering, and contextual understanding. We explore the historical evolution of QA systems before the introduction of LLMs, followed by an in-depth analysis of datasets, benchmarks, and evaluation metrics that underpin DBQA performance. A key contribution of this work is the categorization of LLMs that can perform DBQA based on linguistic focus, distinguishing between monolingual models tailored for specific languages and multilingual models designed to operate across diverse linguistic contexts. Furthermore, we examine notable LLM architectures, their domain-specific adaptations in areas like healthcare and law, and their performance in handling both structured and unstructured data.
Hate speech represents one of the most significant challenges of modern digital society, particularly due to the pervasive use of social networks and online media. Developing reliable automated detection systems requires high-quality, meticulously curated, and annotated datasets - a task that is especially challenging for low-resource languages, such as Serbian. This paper describes the process of collecting, processing, and labeling textual data aimed at creating a dataset for hate speech detection in the Serbian language, as well as a comparative analysis of Large Language Models (LLMs) in the detection and classification of hate speech. The data were gathered from diverse sources, including social networks and online media platforms, utilizing both automated and manual techniques, as well as web crawling and scraping methods. The resulting dataset comprises 1351 short texts, containing 8029 sentences, annotated into three distinct classes: non-hate speech, offensive speech, and hate speech. Furthermore, hate speech instances were additionally categorized according to relevant types of discrimination in accordance with European Union legal acts. In addition to describing the annotation process, this paper provides a detailed analysis of the dataset, including class distribution, text length, and the most frequent keywords. Subsequently, the research selected five LLMs, which were inferred using prompt engineering with a different design for each approach. The models were evaluated using a literature-informed zero-shot prompting strategy based on detailed category definitions, decision criteria, and structured output constraints, implemented through single-stage, two-stage, and paragraph-level prompting settings. The selected models represent recent and widely used open-source general-purpose LLMs available through the Ollama platform, which was chosen to enable local, reproducible, and privacy-preserving evaluation on consumer hardware. Additional comparative experiments were also conducted using few-shot prompting, a general-purpose LLM, and the supervised BERTić model, and the publicly available bcms-bertic-frenk-hate classifier as external baselines. These LLMs were then employed for the detection and classification of hate speech, utilizing an ensemble strategy. The presented results highlight the complexity and diversity of hate speech in Serbian online communication, demonstrating a high detection accuracy of 86.7% achieved with the LLaMA 3 model, followed by Qwen3 (82%) for the two-stage sentence-level pipeline. For paragraph-level analysis, Qwen achieved the highest detection accuracy of 80%, slightly better than LLaMA 3 (77.3%). Ensemble of all five LLMs achieved comparable results to the best selected models, achieving 81.4% in two-stage sentence-level and 79.1% in paragraph-level analysis.
This paper presents a case study on the integration of educational games and gamified assignments in an undergraduate Artificial Intelligence (AI) course. The course leverages interactive, modular Python-based environments to support the teaching of core AI concepts such as search algorithms, constraint satisfaction problems, game theory algorithms, and machine learning. These environments are used through lectures, tutorials, and lab assignments to reinforce theoretical knowledge through problem-solving in simulated scenarios. The research spans 5 academic years and includes the development of 15 rich user interface simulations specifically designed for this course. Quantitative data collected shows a strong positive correlation between participation in assignments designed as educational games and higher final grades. Students who completed these assignments consistently outperformed their peers who did not, suggesting that gamified learning enhances both engagement and outcomes. Additionally, assignment participation has increased year by year, indicating rising student interest and perceived value. The study indicates a strong association between the implementation of game-based learning and enhanced conceptual understanding, while also appearing to motivate students to take a more active role in their education. These findings contribute to growing evidence supporting gamification as an effective strategy in AI education.
This paper presents a document-based question-answering (DBQA) system utilizing a large language model (LLM) to assist students in retrieving relevant information from official university documents. The system effectively processes machine-readable documents by leveraging pre-trained multilingual embeddings and a transformer-based QA model while supporting both Serbian and English queries. The experimental setup focused on extracting relevant context using cosine similarity and answering queries using a fine-tuned QA pipeline, achieving an average accuracy of 84.16
Propaganda is a technique by which people manipulate information to direct the behavior of an individual or group, with the aim of provoking a desired reaction. Propaganda is strongly expressed in the spread of religion, political parties, and autocratic governments. In this paper, the most common propaganda techniques are presented, which propaganda implementers can use in shaping and realizing their messages. A certain content of short texts published on the portals of media companies in Serbia was collected. Then, using machine learning techniques, a model was developed, which was trained to classify the text as propaganda or not propaganda. This research included texts that were published on the portals RTS, Politika, B92, Blic, N1, and Danas in 2023. The texts were collected in real-time, because some media portals do not have an archive.
This research presents a practical application of advanced natural language processing techniques to understand people’s feelings during the global Covid-19 pandemic, using a set of big data of over 547 thousand tweets. Companies often use sentiment analysis to process comments, product usage, social media posts, and more, in order to better understand user needs and preferences. In this research paper, aspect-based sentiment analysis is applied as one of the most recent and advanced subtypes of sentiment analysis. Aspect-based sentiment analysis is a modern natural language processing technique that does not perform sentiment detection at the level of the entire input text but individually over all the aspects detected in it. Therefore, this technique enables the precision of analyzing the user data set and making concrete conclusions about people’s feelings. This research aims to develop a software infrastructure for further work in natural language processing using the aspect-based sentiment analysis technique. The proposed process flow and data handling methods, as defined in this research, are designed to be easily adaptable to other data sets with minimal modifications.
Student hackathons have emerged as a dynamic educational mechanism for fostering creativity and accelerating the adoption of new technologies among Computer Science and Software Engineering students. First introduced in university settings in the early 2000s, hackathons have evolved into highly engaging events where students compete to develop innovative software or hardware solutions within a limited time frame. The competitive spirit inherent to hackathons resonates strongly with today's engineering students, motivating them to push boundaries, solve real-world problems, and collaborate effectively under pressure. Participation in hackathons enhances students' technical skills, promotes teamwork, and deepens their understanding of modern tools, frameworks, and platforms not always covered in formal curricula. Over the past five years, hackathons have gained remarkable popularity in Serbia, especially at state faculties and through active student organizations. Many of these events are organized in partnership with globally recognized ICT companies, providing students with valuable exposure to industry standards and expectations. At the University of Belgrade, the students' programmers' club, within the Google Developer Group on Campus program, has successfully hosted numerous hackathons that have become key platforms for skill development and innovation. These events not only contribute to academic growth but also significantly enhance students' employability and entrepreneurial thinking. Through hands-on experience and interaction with industry mentors, participants often gain clarity about their professional interests and career paths. Hackathons thus serve as bridges between university education and real-world software engineering practice, preparing students for both modern studies and the tech industry.
The problem this research addresses is detecting hate speech (HS) in texts in Serbian on the Internet. Detecting and reducing HS is crucial for the safety and well-being of individuals, as it can otherwise lead to real-world harm and tragedies. Traditional methods of manually monitoring online content are time-consuming, costly, and ineffective in dealing with the vast amount of usergenerated content. Therefore, there is a need for automated tools that can efficiently detect and prevent HS, which constitutes the primary goal of this research. In this paper, we describe the methods of data collection that we intend to implement, present a short analysis of the existing solutions, and provide the conceptual solution of the model.
Legacy software, particularly from the 1990s, often becomes obsolete due to aging hardware and outdated software environments. Traditionally, software modernization required extensive manual effort, involving reverse engineering, code rewriting, and re-architecting. However, advancements in large language models (LLMs) have introduced new possibilities for automating software translation and modernization. This paper explores the feasibility of using LLMs for modernizing 90s-era Windows applications, specifically migrating legacy C and C[Formula: see text] code to Python. Our methodology includes decompilation, source code analysis, automated translation using ChatGPT, and user interface reconstruction. We empirically evaluate three software projects by analyzing LLM-based translation accuracy across different code structures, including algorithmic logic, file handling, and graphical interfaces. Results indicate that while LLMs achieve high translation accuracy [Formula: see text] for structured code, challenges persist in handling decompiled code and user interface generation. The study provides insights into the effectiveness and limitations of LLMs in real-world software renovation, offering guidelines for leveraging machine learning in legacy system modernization. These findings contribute to both academic research and practical applications, suggesting a pathway for cost-effective and scalable legacy software migration.
With a rapid development of artificial intelligence in healthcare, machine learning and federated learning has emerged as critical technologies for improving decision-making, diagnostics and personalized treatments. This paper presents a comprehensive comparison between traditional machine learning and newer federated approach, focusing on their applications in healthcare. Even though machine learning provides great efficiency in resolving complex tasks in healthcare, it faces great challenges such as data privacy, security and communication cost. In contrast, federated learning offers a decentralized approach which resolves these limitations and allows healthcare institutions to collaboratively train models across multiple devices and locations without sharing sensitive patient data. The experiment conducted in this study utilized a medical dataset to predict whether an individual is an alcohol consumer or not. A range of metrics, including loss, accuracy, precision, recall, AUC, and F1 score, were employed to evaluate and compare the performances of both approaches. By leveraging this technique, federated learning addresses privacy concerns, while maintaining high model performances across distributed dataset. Experiments conducted on healthcare datasets highlight the strengths, challenges and limitations of both machine and federated learning in practical, real-world scenarios.
Today, a person can be considered fully digitally literate if they know how to use and integrate ready-made artificial intelligence (AI) tools. The use of AI tools is becoming increasingly common in students' learning processes. However, learning programming can be challenging and exhausting, especially for younger learners. In this research, a software system was developed to integrate multiple AI tools to facilitate the learning process. The system includes tools for speech and text processing, program code generation, code testing, and result verification. Through a user-friendly software interface, users can define a problem or programming task using speech. The software then converts the speech into text using the Whisper AI API, which is subsequently processed by the GPT-3.5 Turbo and Claude AI APIs to generate program code. Once the program code is generated, it undergoes a series of tests, including parallel testing on the LeetCode platform. Users then compare the obtained results and manually complete a survey evaluating both external tools. One key research requirement was for the software system to accept input data in Serbian, a language with limited resources and complex grammatical rules. This made it difficult to find a suitable AI tool for accurate speech-to-text transformation. The system was tested with speech in both English and Serbian but supports many additional languages thanks to the powerful Whisper AI API. The implemented system is modular and easily extensible with new APIs, making it applicable to other areas of education beyond programming.
Visual attention i.e. the automatic selection of the most important information within a visual stimulus - is a topic within computer vision that attracts considerable scholarly interest. In this study, the focus was on examining the complex relationship between covert (implicit, hidden) and overt (explicit, observable) attention and human memory through a memorability experiment on static images. Thirty images for the experiment were selected from the FIGRIM [1] dataset. The collected eye-tracking data were first used to compute fixation maps and visual saliency maps, and then the IOVC (Inter-Observer Visual Congruency) metric; together with the percentage of successful image memorization (the memorability score), these served as the key parameters of the study. A significant correlation between the IOVC score and the memorability score indicated the possibility of automating the prediction of whether viewing an image would lead to successful memorization based solely on its fixation map. Inspired by this insight, we first trained a machine learning model on the collected viewing data to infer, from a fixation map, which of the 30 images had been viewed. The goal was to train a classifier capable of distinguishing images using only their viewing (eye-tracking) data. The trained model achieved close to 90% accuracy in determining whether an arbitrary fixation map would lead to successful memorization of the given image. A web application that could be used for commercial use for memorability prediction was developed.
Hate speech among children in primary and secondary schools represents a widespread and concerning phenomenon. It is unrealistic to expect young individuals to refrain from discriminatory behavior if they are regularly exposed to hateful rhetoric in their environment. This research focuses on the design and development of a web-based application for the detection and classification of hate speech in the Serbian language, with the broader goal of supporting efforts to reduce hate speech among school-aged children. Serbian, as a South Slavic language with limited natural language processing resources, poses additional challenges for computational analysis compared to globally dominant languages due to the scarcity of publicly available datasets. The implemented software tool leverages generative artificial intelligence, specifically state-of-the-art large language models including GPT-4o, Claude 4, Gemini 2.5-Pro, Llama 3, and DeepSeek-R1. The tool is intended for use in educational contexts in Serbia, providing functionality for the analysis of textual content prior to its publication on internet portals and social media platforms. Key features include the automatic detection of hate speech and the generation of revised, nonhateful versions of the text while preserving its original intent.
In this paper, we describe the possibility of using large language models (LLMs) in education through the development of a modern web system. A web-based platform employing LLMs to produce questions and answers spanning diverse domains, drawing upon Serbian Wikipedia as its primary knowledge repository. Leveraging LLMs for question-and-answer generation represents a novel methodology in quiz development. Additionally, incorporating Wikipedia enhances the accuracy and pertinence of the generated questions, thereby enriching the overall user experience.
The rapid increase in the number of Internet of Things (IoT) devices has raised security concerns due to their simplistic architectures. Attacks on these devices can compromise user privacy and disrupt critical infrastructure. This paper focuses on selecting key features from the UNB CIC IOT 2023 dataset to improve network attack classification success using machine learning algorithms and evaluates the effectiveness of existing network monitoring tools in detecting these features. The XGBoost algorithm achieved the highest F1 score of approximately 0.93.
In natural language processing, data acquisition and preprocessing techniques are significant for experiments involving training models on cleaned data. This paper describes the formation of a dataset of ugly and derogatory words in the Serbian language and the development of a web extension to analyze texts in which such words appear and then their censoring. Developing such an add-on is essential in creating a safer and healthier digital environment for young users who are just developing their Internet habits and adults who want to avoid unwanted content. It is crucial to point out that there are few software tools for natural languages with limited resources, such as the Serbian language. This software could analyze the frequency of certain foul words on different sites, which would contribute to a better understanding of the impact of such language on users.
A study on facial keypoints recognition using five deep learning models is presented in this paper, focusing on emphasizing the significance of striking the right balance between model complexity and training data volume. Three convolutional neural networks - LeNet-5, AlexNet, and VGG-16 - are also included in the models that are being presented, along with two custom architectures. The first architecture has a very simple design, while the second one has a regular depth. The study utilizes a small publicly available dataset and focuses on the relationship between model complexity and accuracy in facial keypoints detection. While the more complex custom Neural Network model achieved high accuracy in facial keypoints identification, the simpler custom Neural Network model could not achieve high performance.
Od osnivanja Odseka za softversko inženjerstvo na Elektrotehničkom fakultetu u Beogradu 11. maja 2004. godine pa sve do danas, ovaj Odsek je rastao i razvijao se paralelno sa IT industrijom Republike Srbije i matičnim Elektrotehničkim fakultetom. Akreditovan je četiri puta, 2008, 2013, 2017. i 2024. godine. Na njemu je diplomiralo 932 studenta, od kojih su sedmoro doktorirali na istom fakultetu, a devetoro na prestižnim univerzitetima kao što su MIT, EPFL, ETH Cirih i drugi. Broj studenata koji se upisuju je od početnih 43 porastao na 180 po važećoj akreditaciji. U ovom radu analizirani su uslovi pod kojima je Odsek za softversko inženjerstvo nastao, unapređenja studijskog programa i akreditacione parametre kroz četiri ciklusa akreditacije, a prikazana je i saradnja sa IT kompanijama, kroz pregled obavljenjih stručnih praksi studenata. Koristeći ChatGPT analizirane su teme i tehnologije u kojima su rađeni diplomski radovi, na osnovu naslova tih diplomskih radova.
Sentiment analysis, a pivotal aspect of natural language processing (NLP), offers profound insights into public sentiment from vast swathes of unstructured textual data. This paper presents an empirical investigation into the applicability and effectiveness of the Bidirectional Encoder Representations from Transformers (BERT) algorithm for sentiment analysis, particularly focused on product reviews. The research delves into the nuances of consumer language expressions and evaluates the capacity of BERT to accurately classify sentiment in a large-scale dataset of food product reviews. The results achieved through this research are significant, with the fine-tuned BERT model demonstrating high accuracies, indicating its robustness and suitability for the sentiment classification task. In addressing the challenges posed by the varying lengths of consumer reviews, this study offers a methodological analysis for selecting the optimal maximum sequence length (max_seq_length) parameter within BERT’s framework. A critical balance is achieved between computational efficiency and the comprehensive inclusion of informative content within the reviews. Furthermore, the paper confronts the prevalent issue of class imbalance in sentiment analysis datasets by employing a weighted loss function during the training of BERT. This technique ensures equitable representation and consideration of all sentiment classes, enhancing the model’s accuracy and fairness.
This research paper explores the efficacy of WebAssembly technology in enhancing the performance of web applications, with a focus on graphic-intensive domains such as video games, simulations, and image processing. WebAssembly, a binary instruction format designed for browsers, offers performance comparable to desktop applications by executing code within a secure, low-level virtual machine environment. The study further examines the integration of WebAssembly into various programming languages such as C/C++, C#, and Rust, highlighting its cross-platform capabilities and efficient memory management. Through the development of a website that hosts retro-themed video games, the paper assesses performance across different platforms, demonstrating that while unrestricted desktop platforms show higher speeds, regulated environments exhibit negligible performance discrepancies. The findings suggest that while WebAssembly substantially boosts web application performance, opportunities for further enhancements in areas such as garbage collection, debugging, and JavaScript integration remain. This research underscores WebAssembly's potential to revolutionize web application development, providing a robust framework for future advancements in complex web-based applications.
Veljko M. Milutinovic合作论文数Department of Computer Science and Information Technology, School of Electrical Engineering, University of Belgrade5