In the paper we describe our work in using the Source engine from the game Half Life 2 to develop a serious game Serious Gordon. Serious Gordon is used to teach principals of food safety and food hygiene in a restaurant kitchen environment. The target users of Serious Gordon are students of catering in the DIT Faculty of Tourism and Food. The paper describes the formation of the Serious Gordon team and the development of a story for the game. It continues with a description of the features from Half Life 2 that we retained and those that we removed for the game. We then describe the process we undertook to implement the game including a description of the tools we used. We conclude with a brief evaluation of the project and present future work.

In many real-world classification problems the concept being modelled is not static but rather changes over time a situation known as concept drift. Most techniques for handling concept drift rely on the true classifications of test instances being available shortly after classification so that classifiers can be retrained to handle the drift. However, in applications where labelling instances with their true class has a high cost this is not reasonable. In this paper we present an approach for keeping a classifier up-to-date in a concept drift domain which is constrained by a high cost of labelling. We use an active learning type approach to select those examples for labelling that are most useful in handling changes in concept. We show how this approach can adequately handle concept drift in a text filtering scenario requiring just 15% of the documents to be manually categorised and labelled.

An increasingly important area in supervised incremental learning is learning in the presence of changing concepts. Research into concept drift is hampered by the lack of availability of controllable ‘real life’ datasets. In this paper we propose an approach for generating real life data over which we have control of the concept and can generate data exhibiting different types of concept drift. The approach uses a 3-D driving game to produce a data stream of instances describing how to drive around a track. The classification problem is learning the driving technique of the driver, which can be affected by changes in the driving environment causing changes to the concept. The paper gives illustrations of different types of concept drift and how standard concept drift handling techniques can adapt to the concept drift.

Machine learning has been successfully applied to a wide range of prediction problems, yet its application to data streams can be complicated by concept drift. Existing approaches to handling concept drift are overwhelmingly reliant on the assumption that it is possible to obtain the true label of an instance shortly after classification at a negligible cost. The aim of this thesis is to examine, and attempt to address, some of the problems related to handling concept drift when the cost of obtaining labels is high. This thesis presents Decision Value Sampling (DVS), a novel concept drift handling approach which periodically chooses a small number of the most useful instances to label. The newly labelled instances are then used to re-train the classifier, an SVM with a linear kernel, to handle any change in concept that might occur. In this way, only the instances that are required to keep the classifier up-to-date are labelled. The evaluation of the system indicates that a classifier can be kept up-to-date with changes in concept while only requiring 15% of the data stream to be labelled. In a data stream with a high throughput this represents a significant reduction in the number of labels required. The second novel concept drift handling approach proposed in this thesis is Confidence Distribution Batch Detection (CDBD). CDBD uses a heuristic based on the distribution of an SVM’s confidence in its predictions to decide when to rebuild the classifier. The evaluation shows that CDBD can be used to reliably detect when a change in concept has taken place and that concept drift can be handled if the classifier is rebuilt when CDBD signals a change in concept. The evaluation also shows that CDBD obtains a considerable labels saving as it only requires labelled data when a change in concept has been detected. The two concept drift handling approaches deal with concept drift in a different manner, DVS continuously adapts the classifier, whereas CDBD only adapts the classifier when a sizeable change in concept is suspected. They reflect a divide also found in the literature, between continuous rebuild approaches (like DVS) and triggered rebuild approaches (like CDBD). The final major contribution in this thesis is a comparison between continuous and triggered rebuild approaches, as this is an underexplored area. An empirical comparison between representative techniques from both types of approaches shows that triggered rebuild works slightly better on large datasets where the changes in concepts occur infrequently, but in general a continuous rebuild approach works the best.

Data generated from naturally occurring processes tends to be non-stationary. For example, seasonal and gradual changes in climate data and sudden changes in financial data. In machine learning the degradation in classifier performance due to such changes in the data is known as concept drift and there are many approaches to detecting and handling it. Most approaches to detecting concept drift, however, make the assumption that true classes for test examples will be available at no cost shortly after classification and base the detection of concept drift on measures relying on these labels. The high labelling cost in many domains provides a strong motivation to reduce the number of labelled instances required to detect and handle concept drift. Triggered detection approaches that do not require labelled instances to detect concept drift show great promise for achieving this. In this paper we present Confidence Distribution Batch Detection, an approach that provides a signal correlated to changes in concept without using labelled data. This signal combined with a trigger and a rebuild policy can maintain classifier accuracy which, in most cases, matches the accuracy achieved using classification error based detection techniques but using only a limited amount of labelled data.

In many real-world classification problems the concept being modelled is not static but rather changes over time - a situation known as concept drift. Most techniques for handling concept drift rely on the true classifications of test instances being available shortly after classification so that classifiers can be retrained to handle the drift. However, in applications where labelling instances with their true class has a high cost this is not reasonable. In this paper we present an approach for keeping a classifier up-to-date in a concept drift domain which is constrained by a high cost of labelling. We use an active learning type approach to select those examples for labelling that are most useful in handling changes in concept. We show how this approach can adequately handle concept drift in a text filtering scenario requiring just 15% of the documents to be manually categorised and labelled.

This paper will describe the development of Serious Gordon, an interactive digital game developed to tech the basics of kitchen food safety to workers in industries dealing with food. The motivations driving the development of the game will be described as will the development process itself. An initial evaluation of the game, from both a technical and pedagogical point of view, will be presented as will conclusions on the viability of using a commercial game engine for the purpose of developing educational games.

In the paper we describe our work in using the Source engine from the game Half Life 2 to develop a serious game Serious Gordon. Serious Gordon is used to teach principals of food safety and food hygiene in a restaurant kitchen environment. The target users of Serious Gordon are students of catering in the DIT Faculty of Tourism and Food. The paper describes the formation of the Serious Gordon team and the development of a story for the game. It continues with a description of the features from Half Life 2 that we retained and those that we removed for the game. We then describe the process we undertook to implement the game including a description of the tools we used. We conclude with a brief evaluation of the project and present future work.

We aim to investigate methods balancing exploitation with exploration in active learning to improve the performance of uncertainty sampling. Two exploration guided sampling methods are compared to uncertainty sampling on various real-life datasets from the 2010 Active Learning Challenge. Our initial experiments seems to indicate that combining exploration with uncertainty sampling improves performance on certain datasets but not all.

This paper proposes using exploration guided approaches to select both informative and representative instances to present for labelling in an active learning process.