2D gestures have been extensively examined on surface and interactive tabletop computing, both in the context of training of predefined datasets and "in the wild". The same cannot be said for 3D gestures, however. The current literature does not address what is happening above the tabletop interfaces, nor does it address the semantics.

This paper proposes gesture performance as one main channel for assessing collaboration skills, while multiple users solve a problem collaboratively on a tangible user interface. Collaborative problem solving incorporates two dimensions, complex problem solving and collaboration. Thus, the technology-based assessment of collaborative problem solving includes assessing both problem solving and collaboration skills. Particularly, for assessing collaboration skills, we consider gesture performance as an important indicator. We differentiate between physical 3D mid-air gestures and manipulative gestures; for the latter, we developed a gesture recognition application using Kinect. The method we follow for object and gesture recognition is to merge the logging files from our tangible interface software framework (object recognition) with the Kinect log files (gesture recognition) in one file. The application can analyze the number of object manipulations with respect to timing axis, subject/participant, and handedness.

Competency self-assessment for Performance Appraisal is receiving increasing attention from both researchers and practitioners. Nevertheless, the accuracy and supposed legitimacy of this type of assessment is still an issue. In the context of an industrial use case, we aim to develop and validate a computer-based competency self-assessment technology able to import any type of competency document (for performance appraisal, training need identification, career guidance) and generate semi-automatically self-assessment items. Following the model of Appraisal Effectiveness designed by Levy and Williams, our goal was to build an effective tool meaning that several perspectives must be taken into account: psychometric, cognitive, psychological, political and the reaction’s perspective. In this paper, we will only focus on one specific psychometric property (interrater reliability between an employee and its supervisor). According to a specific rating process and format, our Cross Skill™ technology showed promising results related to interrater reliability in a use case with bank officers and their supervisor.

In our research within the “Gestures in Tangible User Interfaces” project we examine how gestural and speech behaviour while solving a collaborative problem on tangible interfaces has an effect on 21st Century skills. This paper presents an explorative case study where 15–22 year old pupils of a public school in Luxembourg solved a collaborative problem using a tangible user interface (TUI). The main goal of the study was to investigate the frequency and kinds of gestures and the link between the usage of gestures with collaboration and problem solving. At the end of the study, participants had to fill in three standard post-test questionnaires about workload and usability, the results of which are presented here and show that the use of a TUI as an educational medium was regarded as a straightforward and simple solution that many people could learn to use very quickly. This promising result is in line with our research vision, which is to include TUIs in a collaborative setting educational environment.

Test authors can generate test items (semi-) automatically with different approaches. On the one hand, bottom-up approaches consist of generating items from sources such as texts or domain models. However, relating the generated items to competence models, which define required knowledge and skills on a proficiency scale remains a challenge. On the other hand, top-down approaches use cognitive models and competence constructs to specify the knowledge and skills to be assessed. Unfortunately, on this high abstraction level it is impossible to identify which item elements can actually be generated automatically. In this paper we present a hybrid process which integrates both approaches. It aims at securing a traceability between the specification levels and making it possible to influence item generation during runtime, i.e., after designing all the intermediate models. In the context of the European project EAGLE, we use this process to generate items for information literacy with a focus on text comprehension.

Industry 4.0 is a coordinated push for automation in Smart Factories and other Cyber-Physical Systems (CPS). The increasing complexity of frequently changing production environments challenges shop floor workers to perform well. The tasks they work on are getting less routine and ask for continuous knowledge and skills development. For example, the skills portfolio of workers likely requires improved higher-order thinking and decision-making skills. A wide range of research and development efforts already today sets focus on different areas of workplace learning, including performance appraisals, pedagogy and education, technology, and business economics. Bridging the skills gap, however, requires novel user-facing technologies - such as Augmented Reality (AR) and wearables - for human performance augmentation to improve efficiency and effectiveness of staff delivered through live guidance. AR branches out beyond mobile apps with 3D-object superimposition for marketing purposes to rather complex use cases delivered by a rapidly growing innovation ecosystem of hard- and software providers collaborating closely with R&D organisations. This paper provides a first shared vision on how AR can tackle four different challenges related to handling complexity in a CPS environment: develop intelligent assistance systems for learning and performance assessment at the workplace, adapt job profiles accordingly, and last but not least to address also the issue of work-life balance. The paper concludes with an outline of a research roadmap.

This paper presents an intelligent tangible user interface used for learning in a collaborative setting in a secondary school environment. We describe our explorative case study with 48 participants and report the results of a questionnaire filled in after the end of the case study. The questionnaire is related to feedback modalities, such as visual, audio/aural, and haptic, which the participants would like to have offered in the future by the tangible interface or the tangible objects.

This paper reports on an essay-writing study using a technical system that has been developed to generate automated feedback on academic essays. The system operates through the combination of a linguistic analysis engine, which processes the text in the essay, and a web application that uses the output of the linguistic analysis engine to generate the feedback. In this paper we focus on one particular visual representation produced by the system, namely “rainbow diagrams”. Using the concept of a reverse rainbow, diagrams are produced which visually represent how concepts are interlinked between the essay introduction (violet nodes) and conclusion (red nodes), and how concepts are linked and developed across the whole essay – thus a measure of how cohesive the essay is as a whole. Using a bank of rainbow diagrams produced from real essays, we rated the diagrams as belonging to high-, mediumor lowscoring essays according to their structure, and compared this rating to the actual marks awarded for the essays. On the basis of this we can conclude that a significant relationship exists between an essay’s rainbow diagram structure and the mark awarded. This finding has vast implications, as it is relatively easy to show users what the diagram for a “good” essay looks like. Users can then compare this to their own work before submission so that they can make necessary changes and so improve their essay’s structure, without concerns over plagiarism. Thus the system is a valuable tool that can be utilised across academic disciplines.

The construct to assess collaborative complex problem solving has two dimensions: the collaboration and the complex problem solving construct. Both have been defined in the past in the literature, but unfortunately no common model exists. In addition, current assessments lack of authentic tasks which enforce both face-two-face collaboration while solving a complex task. The paper presents a scenario where a task was designed to offer best conditions for assessing collaborative complex problem solving. The principal idea is that the actors play a certain role with specific objectives and different constraints. In addition, different implementations of feedback cues are provided.

. Wissensmanagement (WM) und IT-gestütztes Lernen sind gerade in kleinen Behörden der Öffentlichen Verwaltung (ÖV), wie z.B. in ländlichen Gemeinden, noch ausbaufähig. Am Beispiel des EU-Projekts EAGLE werden Projektergebnisse als Verbesserungsansätze für ein arbeitsprozessorientiertes, IT-gestütztes Lernen vorgestellt. Neuartige Plattform-Features und ihr ÖV-spezifischer Nutzen werden erläutert. Die Ergebnisse der Plattformvalidierung werden vorgestellt. Ferner werden Vorschläge gemacht, wie die Ergebnisse aus EAGLE mit WM und weiteren Wissensquellen der ÖV, wie z.B. der Registra-tur, zu einem Gesamtkonzept mit bereits vorhandenen Fortbildungs-und WM-Ansätzen verbunden werden können.

This article presents a comparative study of the barriers to open e-learning in public administrations in Luxembourg, Germany, Montenegro and Ireland. It discusses the current state of open e-learning of public administration employees at the local government level and derives the barriers to such learning. This paper's main contribution is its presentation of an empirical set of barriers in the four European countries. The results allow informed assumptions about which barriers will arise in the forthcoming use of open-source e-learning technology, particularly open educational resources as means of learning. Furthermore, this study offers a contextualised barrier framework that allows the systematic capture and comparison of challenges for future studies in the field. Other practical contributions include providing advice about open e-learning programmes, systematising lessons learned and addressing managerial implications.

In this work we focus on immediate feedback during technology enhanced assessment. We distinguish two types of feedback: just after answering a test item or just after the completion of a test (cumulative feedback). We identified three challenges related to generation of formative feedback: (1) the lack of information about the user at beginning of the test; (2) the identification of features for the feedback generation on the item level, (3) generation of formative cumulative feedback from limited contextual information. We approach these challenges by creating a user model incrementally from observed user behavior. The conceptual model is validated in an e-learning platform EAGLE targeting information literacy, ICT literacy, and change management in an adult professional learning environment.

This paper presents some implications about designing a collaborative problem solving task on a tangible user interface (TUI) for gesture analysis. We present our initial work within the European Horizon 2020 Marie Curie project Gestures in Tangible User Interfaces (GETUI). This project involves mainly creating a gesture taxonomy used in relation to a tangible tabletop. A preliminary user study showed that gesturing encourages the use of epistemic actions by lowering cognitive load. Our current user studies will enhance 21st Century skills and provide insights about the impact of gestures on collaboration, while also identify cultural differences of gestures.

In this paper we present the need for analyzing (hand) gestures in learning environments and particularly in collaborative problem solving tasks. Based on experimental user studies, we analyze gestures and their impact on technology-based assessment, 21st Century skills as well as on collaboration and cognition.

The goal of this paper is to define relevant barriers to the exchange of Open Educational Resources in local public administrations. Building upon a cultural model, eleven experts were interviewed and asked to evaluate several factors, such as openness in discourse, learning at the workplace, and superior support, among others. The result is a set of socio-cultural factors that shape the use of Open Educational Resources in public administrations. Significant factors are, in this respect, the independent choice of learning resources, the spirit of the platform, the range of available formats and access to technologies. Practitioners use these factors to elaborate on the readiness of public administrations towards the use of open e-Learning systems. To academic debates on culture in e-Learning, the results provide an alternative model that is contextualized to meet the demands of public sector contexts. Overall, the paper contributes to the lack of research about open e-Learning systems in the public sector, as well as regarding culture in the management of learning and knowledge exchange.

This work addresses challenges related to error annotation of conceptually oral learner language such as instant messaging. The analysis is based on a corpus of German longitudinal native/non-native speaker instant messaging dyadic conversations. We show that deviations from language standard in instant messaging can be caused not only by lack of knowledge or high production pace, but also by speakers’ competence in selection of interactional resources for the regulation of the social closeness, leading in this case to two contradicting identities of the same speaker: a competent language learner and a competent instant messaging user. We discuss the consequences from the perspective of language understanding and automated error correction in chat.

“Tomb of Osiris” is a collaborative puzzle-based game on a Tangible Tabletop, with the aim to gamify the MicroDYN approach. MicroDYN is an approach to assess complex problem solving skills through tasks within microworlds. Gamifying MicroDYN proposes new solutions for fostering collaboration and maintaining users’ motivation during assessment. This paper describes the game design, and in particular the different design decisions taken in order to support the MicroDYN procedure, game flow, collaboration, and tangible interactions. By following the design-based research approach, the project aims at exploring and manifesting guidelines for fostering collaboration and motivation in a collaborative game-based MicroDYN scenario.

The chapter introduces the current state of the art of technology for E-Assessment for Learning. A large amount of literature is reviewed with respect to formative assessment, feedback and E-Assessment for Learning. Three case studies introduced single innovative approaches of assessment for learning. These include details of OpenEssayist which is a real-time learning analytics tool using natural language processing techniques to give students feedback on draft essays. A mobile notification system together with automatic item generation techniques for e-assessment systems are also discussed.