Technology-mediated learning (TML) is a major trend in education, since it allows to integrate the strengths of traditional- and IT-based learning activities. However, TML providers still struggle in identifying areas for improvement in their TML offerings. One reason for their struggles is inconsistencies in the literature regarding drivers of TML performance. Prior research suggests that these inconsistencies in TML literature might stem from neglecting the importance of considering the process perspective in addition to the input and outcome perspectives. This gap needs to be addressed to better understand the different drivers of the performance of TML scenarios. Filling this gap would further support TML providers with more precise guidance on how to (re-)design their offerings toward their customers’ needs. To achieve our goal, we combine qualitative and quantitative methods to develop and evaluate a holistic model for assessing TML performance. In particular, we consolidate the body of literature, followed by a focus group workshop and a Q-sorting exercise with TML practitioners, and an empirical pre-study to develop and initially test our research model. Afterward, we collect data from 161 participants of TML vocational software trainings and evaluate our holistic model for assessing TML performance. The results provide empirical evidence for the importance of the TML process quality dimension as suggested in prior literature and highlighted by our TML practitioners. Our main theoretical as well as practical contribution is a holistic model that provides comprehensive insights into which constructs and facets shape the performance of TML in vocational software trainings.

The “perfect” orchestration of training participants, IT and process design is one of the ongoing challenges within blended learning service research and practice. Blended learning services (BLS) offer a great variety of options to design learning processes, overcoming many shortcomings of pure e-learning services and providing better scalability and more advantages for learners than pure face-to-face class teaching. Nevertheless, due to inconclusive results of blended learning design research in the literature, BLS designers can hardly find support for the systematic design of efficient and successful blended learning processes, which would enable a high degree of learning success with a balanced degree of delivery effort. Based on major determinants of BLS processes’ quality, the authors identify, develop, and evaluate design principles for high performance BLS using an action design research approach. They first derive a set of initial design principles, based on insights from literature and own exploratory case studies as well as workshops with experts from the field. They then improve the design principles iteratively in expert workshops as well as apply the design principles in four software training sessions. Finally, they present seven evaluated design principles for BLS, which are the core of a nascent design theory and contribute to a time-efficient and successful BLS delivery. Furthermore, these principles enable practitioners to systematically apply the design knowledge formalized within the principles in order to improve BLS design and delivery.

The evaluation of Technology-Mediated Learning Services (TMLS) has been a major topic for both researchers and practitioners. Nevertheless, inconsistent research results can be observed within TMLS research, leading to the question which determinants and output variables should be used for TMLS evaluation. Therefore, this study examined 85 articles published in peer-reviewed outlets on determinants on learning success and satisfaction within TMLS scenarios. The research was collected from various disciplines, such as IS, psychology, education and business. Thereby, this study systematically examines variables from perspectives such as learner, TMLS provider, process and result. The current state of research in the field of TMLS is presented, focusing on TMLS specific variables which have been researched for (1) at least three times, classified as experimental variables providing research potentials, and (2) at least five times, classified as variables which have been constantly examined. We identified a set of 31 determinants which have a significant or strong significant effect on satisfaction and learning success as well as 34 variables which showed inconsistent effects or provide research potentials. By this means, we identified a significant foundation for TMLS evaluation, as well as inconsistent or not yet extensively examined research results, i.e. future research potentials.

This paper examines the effects of common method variance with respect to the empirical evaluation of technology mediated learning (TML). We argue that the use of self-reported data for the major dependent variable of TML, learning success, is insufficient and a major validity threat of past research results; thus, we examine the effect of common method variance in TML. We are currently conducting a study on the antecedents of learning success measured by three different approaches. We conduct this study with participants of software trainings and collected data on independent and dependent variables of our research model. In addition, we are provided with the objective learning success measures by the provider of the software training. By analyzing the data we are able to investigate how different measurement approaches to learning success impact research findings. Our research is currently in progress, and therefore we are currently not able to provide any empirical findings. The contribution to theory and practice is an assessment of the reliability of self-reported learning success measures and the impact of different measurement approaches for the relationships in a TML model. Besides the on-going discussion in research regarding the concept of learning success measures, our paper is the first that systematically examines the impact of different measurement approaches for learning success in the light of common method variance in TML research.

The article presents a method to the development of a dashboard for large-scale lectures (more than 100 students) based on information derived from educational IT-applications, which play an increasingly important role in the field of university education. Via mobile devices, IT-applications enable students to interact with lecturers as well as their fellow students in large-scale settings, e.g. forum or chat. From a service perspective, lectures can be enhanced by the real-time provision of relevant and useful data. However, data created by the use of IT applications is not yet systematically used to support lecturers’ tasks, i.e,. by providing contents in a well-defined course setting. Feedback comes in the guise of information gathered through the adoption of said devices, e.g,. on what students have understood and which students are intellectually engaged. We are thus in the process of developing a dashboard, which collects information during the lectures to support lecturers’ increasing interaction. We collected design requirements based on experts’ testimonials and relevant literature. These have led to a mock-up which we designed according to literature and expert requirements. We introduce our planned steps to validate the proposed design guidelines through the implementation and evaluation of a proof-of-concept prototype dashboard.

Technology mediated learning services (TMLS) play an important role for software on-the-job-training due to increasing cost pressure for on-the-job-training and increasing demand for mobile learning solutions. As there is no structured approach to systematically deliver productive TMLS this research study creates a holistic reference model of a productivity-optimized TMLS delivery process. This productivity-optimized model focuses on efficient use of resources at a constant or increasing TMLS outcome by a set of process models and design guidelines for TMLS providers. It is developed within a design research setting and derives existing knowledge from the literature and three in-depth-case studies with training providers for software applications. Consequently, we build a reference model that supports the following activities of technology oriented learning: training preparation, training delivery within the classroom, training delivery through internet and training evaluation under consideration of various stakeholders, such as the TMLS provider, trainer, TMLS participant and the participants company.

Technology-mediated Learning Services (TMLS) increasingly play an important role in the vocational training service industry. However, TMLS success is assessed inconsistently within literature. To assess TMLS success, this study focuses on a TMLS productivity perspective, i.e. a systematic inspection of the input, process and output perspective. Consequently, we will develop a TMLS productivity measurement model to assess TMLS from a holistic perspective. In this paper, we present the recent status, comprising TMLS constructs from an input-, throughput and output perspective. They were derived and developed from (1) literature, from an (2) expert workshop and from a (3) qsort-pre-test. To bridge the existing research gap in TMLS assessment, we thereby develop a productivity measurement model that seeks to measure TMLS productivity from a holistic perspective. It contains tangible as well as intangible TMLS factors, both from a customer and a service Provider perspective. We propose to evaluate the model by means of a factor analysis to enrich TMLS Research in terms of systematic evaluation and assessment. Furthermore, we provide practitioners with a tool to measure the productivity of their TMLS using both tangible and intangible indicators.

Technology-mediated Learning Services (TMLS) play an increasingly important role in the learning services industry. Despite the fact that the importance of a holistic evaluation of TMLS quality has been highlighted in the literature to derive transferable research results on multiple dimensions, it has not yet been examined. To address this gap, we first synthesize the existing insights on structure, recipients' predisposition, process and results quality of TMLS and develop a comprehensive Approach to measure TMLS quality. Afterwards, we develop our research model and examine the importance of the different constructs of TMLS quality. We rely on findings in the body of literature, a focus group workshop and a card-sorting exercise to develop our TMLS quality model. Thereafter, we collect data from 163 participants of TMLS software-training to empirically evaluate our scale and research model. Our core results are a TMLS Quality model, including a newly developed TMLS process dimension.

This article presents success factors for the delivery of Technology Mediated Learning Services (TMLS) in the field of software training. The latter are a significant component in regards of the introduction of Software in companies. However, very little research exists that collects the factors of various dimensions of TMLS factors of influence from a service perspective, and assesses the effective employment, in particular the influence on the learning success. Factors of influence that affect TMLS are evaluated, derived and expanded by means of an expert Delphi study. An overview of the success factors rated most important is presented, taking the dimensions (1) characteristics of the trainer, (2) promotion of the learning process, (3) learning material (offline and online) as well as (4) organizational influences into account. By means of this estimation, recommendations for advanced training services are deduced. Thereby, to the best of our knowledge, for the first time a comprehensive comparison of multidimensional measures is conducted.

The article presents a method to the development of a dashboard for large-scale lectures (more than 100 students) based on information derived from educational IT-applications, which play an increasingly important role in the field of university education. Via mobile devices, IT-applications enable students to interact with lecturers as well as their fellow students in large-scale settings, e.g. forum or chat. From a service perspective, lectures can be enhanced by the real-time provision of relevant and useful data. However, data created by the use of IT applications is not yet systematically used to support lecturers’ tasks, i.e,. by providing contents in a well-defined course setting. Feedback comes in the guise of information gathered through the adoption of said devices, e.g,. on what students have understood and which students are intellectually engaged. We are thus in the process of developing a dashboard, which collects information during the lectures to support lecturers’ increasing interaction. We collected design requirements based on experts’ testimonials and relevant literature. These have led to a mock-up which we designed according to literature and expert requirements. We introduce our planned steps to validate the proposed design guidelines through the implementation and evaluation of a proof-of-concept prototype dashboard.

Measuring Technology Mediated Learning (TML) success has been and is of great interest to both researchers and practitioners. This article examines multidimensional approaches to measuring learning success, considering IT- and non-IT-supported learning scenarios, examined by researchers from various research disciplines. We explore the current state of research on TML success through a literature review by classifying empirically-oriented articles that were published between 2000 and 2011. Based on a total of 91 articles published in academic journal publications, this paper identifies the relevant research carried out, categorizes and consolidates the research results, and discusses them. The results show that those approaches are most dominant which empirically analyze the impact of a certain type of educational service on a student / participant by means of surveys and structural equation modeling to capture users' responses. No main theoretical basis of the reviewed studies could be identified. Furthermore, opportunities for additional development are identified and future research directions suggested.

Universitare Massenlehrveranstaltungen bieten Dozenten kaum Moglichkeiten mit Studenten zu interagieren. Durch IT-Einsatz wird verstarkt versucht die Interaktion bzw. den Lernerfolg zu verbessern. Dadurch bietet sich die Moglichkeit, die durch die IT-Verwendung entstehenden Informationen zu nutzen. Ziel der vorliegenden Research in Progress-Arbeit ist daher die Identifikation von Informationen, die fur Dozenten zur Beeinflussung des Lernerfolgs geeignet sind. Die Informationen werden als Kennzahlen in ein Performance Dashboards einfliessen. Hierzu wurden in einem Workshop mit Universitatsdozenten bewertete Kennzahlen entwickelt, die die Grundlage fur die Aufbereitung in einem Performance Dashboard bilden. Dozenten soll die Moglichkeit gegeben werden, das Lernverhalten der Studenten nachzuvollziehen und deren Lernerfolg zu verbessern.

Lernerfolg wird durch eine aktive Auseinandersetzung mit Inhalten, die Anwendung von Wissen und Interaktion wie die Teilnahme an Diskussionen gefordert. Diese Aspekte kommen in universitaren Massenveranstaltungen aufgrund der hohen Teilnehmerzahlen jedoch haufig zu kurz. Mobile Endgerate bieten an dieser Stelle das Potenzial, bisherige Strukturen aufzubrechen und neue Interaktionsmoglichkeiten zu schaffen. Im Rahmen eines Pilotprojekts an einer deutschen Universitat wurden mehr als 150 mobile Endgerate, vorwiegend in Form von Tablet Computern, als Leihgerate an Studierende einer Massenlehrveranstaltung in den Wirtschaftswissenschaften ausgegeben. Im Rahmen der Veranstaltung kamen Applikationen zu Teilnehmeraktivierungen mit Hilfe der mobilen Endgerate sowie mobile Lernmaterialien in Form von Videos und Web Based Trainings zum Einsatz. Abschliessend fand eine Evaluation mittels eines Online Fragebogens statt (n = 128), welche die wahrgenommene Interaktivitat und Zufriedenheit der Studierenden mit Lernmaterialien und Teilnehmeraktivierungen erfassen sollte. Die Ergebnisse deuten darauf hin, dass die entwickelten Lernanwendungen insgesamt als Bereicherung der Lernerfahrung wahrgenommen werden. Die Interaktivitat der Lehrveranstaltung wird besonders positiv bewertet, insbesondere das Lernen mit Hilfe eines Tablet Computers steigert bei vielen Studierenden den Spass an der Lehrveranstaltung. Die Neuartigkeit der Arbeit liegt im flachendeckenden Einsatz von mobilen Endgeraten innerhalb und ausserhalb einer Massenveranstaltung. Die Ergebnisse sind fur Dozenten in der universitaren Lehre von praktischer Relevanz und zeigen zugleich wichtige didaktische Herausforderungen beim intelligenten Einsatz von IT in Massenveranstaltungen auf.

Der Trend zum Einsatz von E-Learning und neuerdings Mobile Learning an deutschen Hochschulen scheint ungebrochen. Die rasante technologische Entwicklung erlaubt die Gestaltung immer neuer Lehr-Lern-Arrangements. Der Fokus der meisten Untersuchungen in diesem Bereich liegt dabei auf Zufriedenheit und Lernerfolg der Studierenden. Dabei wird oft die Sicht der Lehrenden vernachlassigt, deren personliche Einstellung zum E-Learning und Bereitschaft zur Anpassung der eigenen Lehrgewohnheiten und Lerninhalte jedoch wesentlichen Einfluss auf die erfolgreiche Neugestaltung der Lehre haben. Im Rahmen eines Pilotprojekts1 an der Universitat Kassel wurden Studierende mehrerer Lehrveranstaltungen flachendeckend mit mobilen Endgeraten, insbesondere Tablet PCs, ausgestattet. Die Dozenten mussten im Zuge dessen ihre eigenen Lehrgewohnheiten anpassen und ihre bisherigen Lehr-Lern-Arrangements umgestalten. Anschliessend wurden vier semi-strukturierte Interviews mit den Lehrenden gefuhrt. Die Ergebnisse weisen auf mogliche neue Einsatzszenarien aber auch neuartige Herausforderungen fur Lehrende hin und geben einen Uberblick uber den wahrgenommenen Aufwand aus Dozentensicht. Damit ist diese Studie eine der wenigen, die sich mit der Dozentensicht bei der Einfuhrung neuer Technologien beschaftigt. Sie zeigt wesentliche Aspekte der erfolgreichen Einfuhrung von Blended Learning und Mobile Learning im universitaren Lehrbetrieb auf und bietet Ansatze zur weiterfuhrenden Erforschung von Massnahmen zur Weiterentwicklung der universitaren Lehre.

The study aims to develop a comprehensive set of input and output factors for educational services. In the frame of the study, a set of input, throughput and output factors is presented. These factors were collected in a workshop with lecturers having a long-time experience in teaching, since the authors assume that they have a great amount of implicit knowledge about the productive delivery of educational services. So far, no further refinement on the different factors was conducted, thus an actual usage of the taxonomy is not possible. Hence, it is necessary, to find a comprehensive set of factors, which helps to operationalize the factors which consequently can be converted into an applicable productivity measurement model. Based on the workshop results, further details to a beforehand developed productivity model is added, which combines and classifies input and output factors. The advanced productivity model will allow classifying existing learning scenarios and enables scientists to derive appropriate measurement tools to measure the productivity of educational services. Based on the workshop findings an approach for an learning scorecard is presented.

Designing performance indicators for measuring the value of IT supported processes is an important and recurring task in controlling and management. Involving different stakeholders and perspectives in the development of an evoked set of performance indicators is the key to improve the quality and acceptance of performance measures. We apply an approach for designing repeatable processes for the development of such performance indicators as high-value collaborative tasks using techniques of the collaboration engineering. We show how this collaboration engineering approach can be designed (building on the collaboration process design approach by Kolfschoten et al. (Ko06a)) and how it needs to be extended for designing a performance measurement tool for IT applications. The case describes the development process within a multinational industrial company for a reporting application. It shows that the approach is promising to avoid classical problems of the performance measurement, e.g. conflicting interests of various stakeholders. The developed concept provides a collaborative approach for companies to develop key performance indicators in a politically challenging environment and to generate a valid performance measurement tool. The results suggest that the outcome of the workshop provides a broad variety of indicators, which represent different aspects of benefits of the application. The use of collaboration engineering approaches for the field of performance measurement delivers promising results and offers interesting options for further research, i.e. standardization potentials for collaborative performance tool development and requirements of corporate environments.