There is a significant gap in Computing Education Research (CER) concerning the impact of Large Language Models (LLMs) in advanced stages of degree programmes. This study aims to address this gap by investigating the effectiveness of LLMs in answering exam questions within an applied machine learning final -year undergraduate course. The research examines the performance of LLMs in responding to a range of exam questions, including proctored closed -book and open -book questions spanning various levels of Bloom's Taxonomy. Question formats encompassed open -ended, tabular data -based, and figure -based inquiries. To achieve this aim, the study has the following objectives: Comparative Analysis: To compare LLM-generated exam answers with actual student submissions to assess LLM performance. Detector Evaluation: To evaluate the efficacy of LLM detectors by directly inputting LLM-generated responses into these detectors. Additionally, assess detector performance on tampered LLM outputs designed to conceal their AIgenerated origin. The research methodology used for this paper incorporates a staff -student partnership model involving eight academic staff and six students. Students play integral roles in shaping the project's direction, particularly in areas unfamiliar to academic staff, such as specific tools to avoid LLM detection. This study contributes to the understanding of LLMs' role in advanced education settings, with implications for future curriculum design and assessment methodologies.

This research has three prongs, with each comparing open- and closed-book exam questions across six years (2017-2023) in a final year undergraduate applied machine learning course. First, the authors evaluated the performance of numerous LLMs, compared to student performance, and comparing open and closed book exams. Second, at a micro level, the examination questions and categories for which LLMs were most and least effective were compared. This level of analysis is rarely if ever, discussed in the literature. The research finally investigates LLM detection techniques, specifically their efficacy in identifying replies created wholly by an LLM. It considers both raw LLM outputs and LLM outputs that have been tampered with by students, with an emphasis on academic integrity. This study is a staff-student research collaboration, featuring contributions from eight academic professionals and six students.

While recent approaches to sign language processing have shifted to the domain of Machine Learning (ML), the treatment of NonManual Features (NMFs) remains an open question. The principal challenge facing this method is the comparatively small sign language corpora available for training machine learning models. This study produces a statistical model which may be used in future ML, rules-based, and hybrid-learning approaches for sign language processing tasks. In doing so, this research explores the emerging patterns of non-manual articulation concerning grammatical classes in Irish Sign Language (ISL). The experimental method applied here is a novel implementation of an association rules mining approach to a sign language dataset consisting of NMF and grammatical class data from the Signs of Ireland corpus. Our analysis of association rules has identified patterns between grammatical classes and various non-manual articulations. One such pattern discovery is the strong correlation between various NMFs and depicting verbs. Indeed, this study reports that the less lexicalised a sign is, the more likely it is to use NMFs. Findings from this work will inform future research on NMF treatment in sign language processing, while the statistical model may be utilised by such systems in the future.

The detection of bacterial antibiotic resistance phenotypes is important when carrying out clinical decisions for patient treatment. Conventional phenotypic testing involves culturing bacteria which requires a significant amount of time and work. Whole-genome sequencing is emerging as a fast alternative to resistance prediction, by considering the presence/absence of certain genes. A lot of research has focused on determining which bacterial genes cause antibiotic resistance and efforts are being made to consolidate these facts in knowledge bases (KBs). KBs are usually manually curated by domain experts to be of the highest quality. However, this limits the pace at which new facts are added. Automated relation extraction of gene-antibiotic resistance relations from the biomedical literature is one solution that can simplify the curation process. This paper reports on the development of a text mining pipeline that takes in English biomedical abstracts and outputs genes that are predicted to cause resistance to antibiotics. To test the generalisability of this pipeline it was then applied to predict genes associated with Helicobacter pylori antibiotic resistance, that are not present in common antibiotic resistance KBs or publications studying H. pylori. These genes would be candidates for further lab-based antibiotic research and inclusion in these KBs. For relation extraction, state-of-the-art deep learning models were used. These models were trained on a newly developed silver corpus which was generated by distant supervision of abstracts using the facts obtained from KBs. The top performing model was superior to a co-occurrence model, achieving a recall of 95%, a precision of 60% and F1-score of 74% on a manually annotated holdout dataset. To our knowledge, this project was the first attempt at developing a complete text mining pipeline that incorporates deep learning models to extract gene-antibiotic resistance relations from the literature. Additional related data can be found at https://github.com/AndreBrincat/Gene-Antibiotic-Resistance-Relation-Extraction.

Spelling correction utilities have become commonplace during the writing process, however, many spelling correction utilities suffer due to the size and quality of dictionaries available to aid correction. Many terms, acronyms, and morphological variations of terms are often missing, leaving potential spelling errors unidentified and potentially uncorrected. This research describes the implementation of WikiSpell, a dynamic spelling correction tool that relies on the Wikipedia dataset search API functionality as the sole source of knowledge to aid misspelled term identification and automatic replacement. Instead of a traditional matching process to select candidate replacement terms, the replacement process is treated as a natural language information retrieval process harnessing wildcard string matching and search result statistics. The aims of this research include: 1) the implementation of a spelling correction algorithm that utilizes the wildcard operators in the Wikipedia dataset search API, 2) a review of the current spell correction tools and approaches being utilized, and 3) testing and validation of the developed algorithm against the benchmark spelling correction tool, Hunspell. The key contribution of this research is a robust, dynamic information retrieval-based spelling correction algorithm that does not require prior training. Results of this research show that the proposed spelling correction algorithm, WikiSpell, achieved comparable results to an industry-standard spelling correction algorithm, Hunspell.

Applied User Data Collection and Analysis Using JavaScript and PHP is designed to provide the technical skills and competency to gather a wide range of user data from web applications in both active and passive methods. This is done by providing the reader with real-world examples of how a variety of different JavaScript- and PHP-based libraries can be used to gather data using custom feedback forms and embedded data gathering tools. Once data has been gathered, this book explores the process of working with numerical data, text analysis, visualization approaches, statistics, and rolling out developed applications to both data analysts and users alike.Using the collected data, this book aims to provide a deeper understanding of user behavior and interests, allowing application developers to further enhance web-application development. Key Features: Complete real-world examples of gathering data from users and web environments Offers readers the fundamentals of text analysis using JavaScript and PHP Allows the user to understand and harness JavaScript data-visualization tools Integration of new and existing data sources into a single, bespoke web-based analysis environment

Word frequency has a significant impact on language acquisition and fluency. It is often a point of reference for the teaching and assessing of a language and indeed, as a control for psycholinguistic studies. This paper presents the results of the first objective frequency analysis of lexical tokens from the Signs of Ireland corpus. We investigate the frequency of fully lexical, partly lexical and non-lexical signs in Irish Sign Language as they are presented in the corpus. We confirm the accuracy of the lexical gloss frequency data with a supplementary corpus subset that is tagged for grammatical class and additional insights from previous lexical frequency studies conducted for American Sign Language, Australian Sign Language, British Sign Language and New Zealand Sign Language. This approach has informed us that signers who learn the 100 most frequent signs in Irish Sign Language, will know a third of the language’s vocabulary. This study has found that, in the main, frequency statistics from Irish Sign Language are in line with previous studies and that the text type and annotation strategy can significantly impact results. We found that, without a formalised lexicon, lexical glosses fell short of the requirements for a lexical frequency analysis. However, supported by grammatical class data, frequency data may be reported for symbolic units.

The majority of datasets suffer from class imbalance where samples of a dominant class significantly outnumber the samples available for the minority class that is to be detected. Prediction and classification machine learning models work best when there are roughly equal numbers of each class type. This paper explores sampling techniques that can be used to overcome this class imbalance problem in a cyberbullying context. A newly classified cyberbullying dataset, including detailed descriptions of the criteria used in its classification, was used to examine the feasibility of applying text mining techniques, to automate the detection of cyberbullying text when the dataset shows a significant class imbalance between the positive, cyberbullying, sample and the negative, not cyberbullying, samples. In this paper, we will investigate if oversampling the minority positive class or undersampling the majority negative class affects the performance of a prediction model. A compromise solution where the positive class is partially oversampled, and the negative class is partially undersampled is also examined. Although not strictly a class imbalance solution, sampling using the most frequently observed features was also explored.

Traditionally, statistical methods were used to analyze customer data to gain insight. Recently, data mining techniques have evolved and have been used to analyze customer data. This paper demonstrated how Data Mining techniques can be applied to customer satisfaction analysis of MEA Smartlife, a mobile application (app) developed by Metropolitan Electricity Authority (MEA) of Thailand in order to provide some e-service features to its customers. User satisfaction rating along with demographic profile from 1,446 subjects with diverse backgrounds were collected. Machine learning techniques were then applied to the dataset following the CRISP-DM methodology. Modeling techniques for this study include decision tree, Naive Bayes, and logistic regression as intuitiveness of the model rather than the predictive performance is more important than predicting whether the customers like the app. The resulting models achieved more than 90% accuracy while having a lower level of precision on the negative class. Feature selection techniques help improve overall accuracy and improve the negative class precision. The resulting model indicated ‘ease of use’ is the most important factor in determining whether customers are satisfied or dissatisfied with the app. The payment feature also plays an important role in making customers satisfied with the app.

—Distributed word representation techniques have been effectively integrated into the Information Retrieval retrieval task. The most basic approach to this is mapping a document and the query words into the vector space and calculating the semantic similarity between them. However, this has a bias problem towards documents with different lengths, which rank a small document higher compared to documents with larger vocabulary size. While averaging a document by mapping it into vector space, it allows each word to contribute equally, which results in increased distance between a query and the document vectors. In this paper, we propose that document length normalization should be applied to address the length bias problem while using embedding based ranking. Therefore, we have presented an experiment with traditional Length Normalization techniques over, word2vec (Skip-gram) model trained using the TREC Blog06 dataset for ad-hoc retrieval tasks. We have also attempted to include relevance signals introducing a simple Linear Ranking (LR) function, which considers the presence of query words in a document as evidence of relevancy while ranking. Our combined method of Length Normalization and LR significantly increases the Mean Average Precision up to 47% over a simple embeddings based baseline.

Automatic Search Query Enhancement (ASQE) is the process of modifying a user submitted search query and identifying terms that can be added or removed to enhance the relevance of documents retrieved from a search engine. ASQE differs from other enhancement approaches as no human interaction is required. ASQE algorithms typically rely on a source of a priori knowledge to aid the process of identifying relevant enhancement terms. This paper describes the results of a qualitative analysis of the enhancement terms generated by the Wikipedia NSubstate Algorithm (WNSSA) for ASQE. The WNSSA utilises Wikipedia as the sole source of a priori knowledge during the query enhancement process. As each Wikipedia article typically represents a single topic, during the enhancement process of the WNSSA, a mapping is performed between the user’s original search query and Wikipedia articles relevant to the query. If this mapping is performed correctly, a collection of potentially relevant terms and acronyms are accessible for ASQE. This paper reviews the results of a qualitative analysis process performed for the individual enhancement term generated for each of the 50 test topics from the TREC-9 Web Topic collection. The contributions of this paper include: (a) a qualitative analysis of generated WNSSA search query enhancement terms and (b) an analysis of the concepts represented in the TREC-9 Web Topics, detailing interpretation issues during query-to-Wikipedia article mapping performed by the WNSSA.

This paper describes the development and testing of a novel Automatic Search Query Enhancement (ASQE) algorithm, the Wikipedia N Sub-state Algorithm (WNSSA), which utilises Wikipedia as the sole data source for prior knowledge. This algorithm is built upon the concept of iterative states and sub-states, harnessing the power of Wikipedia's data set and link information to identify and utilise reoccurring terms to aid term selection and weighting during enhancement. This algorithm is designed to prevent query drift by making callbacks to the user's original search intent by persisting the original query between internal states with additional selected enhancement terms. The developed algorithm has shown to improve both short and long queries by providing a better understanding of the query and available data. The proposed algorithm was compared against five existing ASQE algorithms that utilise Wikipedia as the sole data source, showing an average Mean Average Precision (MAP) improvement of 0.273 over the tested existing ASQE algorithms.