This article presents the results of a design workshop that developed scenarios of learning in an imaginary "smart city": Villard-de-Lans (Vercors, French Alps) by exploring suitable methodological approaches to the Smart City Learning Design. The aim of this case study was to propose "glocal" solutions intended to balance the global trends and the expectations of the community of reference (local elements), while distinguishing between access to information and participation in informal learning processes located in a smart city.

In this paper we discuss challenges of usability evaluation of mobile applications. We outline some key aspects of mobile applications and the special characteristics of their usability evaluation that have recently lead to the laboratory vs field discussion. Then we review the current trend and practices. We provide an example of a usability evaluation study from our own background. We conclude with discussion of some open issues of usability evaluation of mobile applications. Author

Abstract Effective and efficient usability evaluation methodologies are required in order to develop and maintain software applications that meet user requirements and expectations. This is particularly relevant to applications that change frequently during their lifecycle . Remote usability evaluation provides such a solution. Numerous approaches and techniques have been proposed to allow remote usability evaluation of software applications, and the purpose of this paper is to categorize and evaluate these approaches. We also discuss relevant issues such as user data collection and analysis. After surveying the proposed approaches and identifying pros and cons, we conclude with promising directions for future research in this area. Keywords: Human Computer Interaction, Usability Evaluation, Remote Usability Evaluation Methods, Remote Usability Evaluation Tools 1. Introduction The rapidly expanding research in software usability evaluation is based on the premise that usable software products increase user satisfaction, effectiveness and efficiency, improve product quality while simultaneously reduce maintenance and support costs [Nielsen (1993), Dix et Al. (1998)]. Usability refers to whether a system can be used with effectiveness, efficiency, and satisfaction with which specified users achieve specified goals in a particular context of use [ISO 9241-11]. Methodologies for software development need to take advantage of user centred design approaches which give extensive attention to the needs and limitations of the users at each stage of the design process [Vredenburg et. Al.(2002)]. Furthermore, a relatively recent trend in web applications which is related to the

Classification problems with uneven class distributions present several difficulties during the training as well as during the evaluation process of classifiers. A classification problem with such characteristics has resulted from a data mining project where the objective was to predict customer insolvency. Using the data set from the customer insolvency problem, we study several alternative methodologies, which have been reported to better suit the specific characteristics of this type of problem. Three different but equally important directions are examined: (a) the performance measures that should be used for problems in this domain; (b) the class distributions that should be used for the training data sets; and (c) the classification algorithms to be used. The final evaluation of the resulting classifiers is based on a study of the economic impact of classification results. This study concludes to a framework that provides the “best” classifiers, identifies the performance measures that should be used as the decision criterion, and suggests the “best” class distribution based on the value of the relative gain from correct classification in the positive class. This framework has been applied in the customer insolvency problem, but it is claimed that it can be applied to many similar problems with uneven class distributions that almost always require a multi-objective evaluation process.

In this paper, we describe a usability evaluation study of a system involving PDAs, designed to be used in a traditional historical/ cultural mu- seum. The system permits collaboration of small groups of museum visitors through mobile handheld devices. The key characteristics of the system are de- scribed first, which include a server and a client component and a tool that permits authoring of new activities. The usability evaluation study that in- volved typical users revealed some of the limitations of the design. The re- ported findings can be of use to practitioners interested in following similar ap- proaches relating to evaluation of mobile technology.

This paper presents Synergo, a collaboration support environment that monitors the activity and permits visualization of various quantitative parameters, like density of interaction, symmetry of the activity, degree of collaboration etc, particularly useful for understanding the mechanics of collaboration. Synergo has been proposed as a testbed for analysis of small-group model-building synchronous interaction.

This paper examines the effect of heterogeneous resources, available to students, during computer-supported collaborative problem solving. A study of collaborative modeling has been conducted in the frame of an authentic educational activity in a secondary school. The students involved were provided with sets of primitive resources of varying degrees of heterogeneity to be used during synchronous computer-mediated modeling activities. Analysis of students' peer interaction and of the produced solutions revealed that, contrary to our expectations, the group with heterogeneous resources produced solutions of similar quality to those of the reference group, although they were more active, they exchanged more messages, they were involved in deeper discussions and collaborated more for building the constituent parts of the solution.

ModellingSpace is a learning environment that allows collaboration of partners, collocated or at a distance in various educational settings. This paper describes the main features of the architecture of the ModellingSpace environment and in particular issues related with coordination and communication during problem solving. The results of an evaluation study of ModellingSpace, which has been recently contacted in order to examine the effect of various levels of locking of objects in the shared activity space are also reported here. Through this study, the effectiveness and the limitations of the proposed architecture are identified and discussed.

In this paper, we present a flexible software environment that facilitates the use of machine learning techniques in power system contingency studies as an alternative to traditional power flow analysis. The architecture of this toolkit, which includes the database repository, and a number of machine learning tools are described. The toolkit approach enables the user to experiment with the predictive powers of various machine-learning tools over various network operating points. The paper covers the findings of a case study performing a sensitivity analysis using the presented software environment.

This paper describes our experience with introduction of synchronous collaborative problem solving activities in the frame of a distance learning computer science undergraduate course of the Hellenic Open University (HOU). Groups of students worked collaboratively at a distance in order to build a flowchart of an algorithm to a given problem. The technological and organization issues involved, the first findings of analysis of peer student interaction during this study, as well as some general implications for distance education are discussed.

Studies of collaborative learning activities often involve analyses of dialogue and interaction as well as analyses of tasks and actors' roles through ethnographic and other field experiments. Adequate analysis tools can facilitate these studies. In this paper, we discuss key requirements of interaction and collaboration analysis tools. We indicate how these requirements lead to the design of new analysis environments. These environments support annotation and analysis of various kinds of collected data in order to study collaborative learning activities. An important characteristic of these tools is their support for a structure of annotations of various levels of abstraction, through which an activity can be interpreted and presented. This can serve as a tool for reflection and interpretation as well as for facilitation of research in collaborative learning.

Feature selection is a process followed in order to improve the generalization and the performance of several classification and/or regression algorithms. Feature selection processes are divided in two categories, the filter and the wrapper approach. The formal is performed independently of the learning algorithm while the later makes use of the algorithm in an iterative way. As [1] describe, the feature weighting algorithms are divided into two categories: the filtering methods and the wrapper methods. The former is a no-feedback, pre-selection approach where the selection of the feature subset is performed independently of the learning algorithm. The later is an iterative method that encapsulates the learning algorithm in the feature selection process.

A model-based approach in design and evaluation of computer-based open problem solving environments is presented in this paper. The functionality of a tool (CMT) that has been developed to support this process is also discussed. A number of models are defined according to the proposed approach: (a) a designer model and (b) user models as deduced from observation of user behaviour during field studies. Identification of discrepancies of the two models can lead to improvements in the design of developed prototypes. Examples of application of this design approach and the CMT tool are included in the paper.

It is believed that computer-supported collaboration at a distance can stimulate learning. An innovative environment that permits real-time collaborative problem solving is described. In particular we study the effect of two alternative coordination mechanisms on the problem solving activity of pairs of students engaged in concept map building. The first mechanism imposes locking of the shared activity board for one student at a time, while the second mechanism allows access of all group members to the shared activity board in a contemporary way. The reported findings are of interest to researchers and practitioners who are involved in the design and study of real-time collaborative learning environments.

An increasing amount of data is collected today during studies in which students and educators are engaged in learning activities using information technology and other tools. These data are indispensable for analysis and evaluation of learning activities, for evaluation of new tools and for students’ meta-cognitive activities. The data can take various forms, including video and audio recordings, log files of computing-related activity, field notes, results of students work in electronic or other forms, activity sheets etc. The need for analysis tools, which can annotate these data, classify them, process them and facilitate their inspection, is of increased importance especially for science education, since the latter involves experimentation and use of laboratory and other equipment that necessitate thorough off-line analysis and evaluation. In this paper we discuss first the key requirements of a new generation of interaction and collaboration analysis tools. We then present how these requirements have lead to the design of a prototype tool, recently developed. This tool can relate and synchronize various streams of field data. An important characteristic of the tool is its support for a multi-layer structure of annotations of various levels of abstraction, through which the activity can be interpreted and presented. This multi-layer representation can be inter-related to the raw field data, and can drive the navigation of the researcher in the activity data. An example of use of this tool for analysis and evaluation of a collaborative problem solving activity is also included.

Computer-supported collaborative problem solving requires new methodological approaches of interaction and problem solving analysis. Usually analysis of collaborative problem solving situations is done through discourse analysis or interaction analysis, where in the center of attention are the actors involved (students, tutors etc.). An alternative framework, called “Object-oriented Collaboration Analysis Framework (OCAF)” is presented here, according to which the objects of the collaboratively developed solution become the center of attention and are studied as entities that carry their own history. This approach produces a view of the process, according to which the solution is made of structural components that are ‘owned’ by actors who have contributed in various degrees to their development. OCAF is based on both actions and dialogues of actors, providing qualitative as well as quantitative indicators of collaboration and solution quality. The paper presents first the framework notation. Examples of its use in analysis of distance groups and face-to-face collaborative activities are provided next, followed by the dimensions of the framework supported analysis for teachers and researchers. Web-based tools supporting the OCAF approach are also presented.

An innovative framework of interaction and collaboration analysis is proposed, jointly with tools to support the process. The proposed framework is based on a collaboration analysis first and individual task analysis subsequently, approach. The objective of this framework is to facilitate understanding of the group’s and individual user’s tasks and goals and associate the artifacts used with usability problems. An innovative aspect of the framework is the association of tasks to artifacts (tools) engaged by the users during the activity. The typical use of this framework is in interactive systems evaluation and design. The framework and the tools functionality are described in the paper. The framework is inspired by the Activity Theory perspective, which recognizes the importance of artifacts, actors and the context in which an activity takes place.

Feature selection is a process of determining the most relevant features of a given problem in order to improve the generalization and the performance of a relevant classification or regression algorithm.This paper focuses on the exploitation of a genetic algorithm following a wrapping iterative approach used to extract an optimal feature subset of a large database containing pollutant concentration measurements. The feature subset is fed to a machine learning algorithm in order to predict the daily maximum concentration of two air pollutants.The encoding problem of the complexity of representation of the features in the genomes is tackled. Results of the experimentation on a specific dataset of an air quality forecasting problem are presented, as well as some proposed alterations on the standard genetic algorithm that guided the process to a mature convergence and gave good solutions for this problem. A modified version of the initial algorithm is presented as well, implemented for the purpose of being compared on an equal basis with other feature selection methods. Two such methods of the filtering type, CFS and ReliefF, are being compared with.The comparative results suggest that the wrapping type technique described in this paper is significantly better in the specific problem at hand, but this conclusion is limited to the machine learning algorithm that the technique uses at its core in the feature selection phase.

Analysis of collaborative problem solving involves analysis of dialogue and interaction, analysis of tasks and social roles through ethnographic and other field studies. Use of tools to facilitate this process can be very useful. We discuss first the key requirements of a new generation of interaction and collaboration analysis tools. We then present how these requirements have lead to the design of prototype tools, recently developed. These tools can relate and synchronize various streams of field data. An important characteristic of the tools is their support for a multilayer structure of annotations of various levels of abstraction, through which the activity can be interpreted and presented.

Analysis of interaction between computer artefacts and learning actors in the context of collaborative problem solving is a tedious process, which needs to be supported by appropriate tools. In this paper we present tools, recently developed, that can support interrelation and synchronization of various streams of field data. A key characteristic of these tools is their support for a multi-level structure of annotations, through which the problem- solving activity can be interpreted and presented. This multi-level representation can be inter-related to the raw field data, and can drive the navigation of the researcher in the activity data. An example of use of these tools for analysis and evaluation of a collaborative problem solving activity is presented in the demo that accompanies this paper. During a field study, data in various forms may be produced. These may often be in the form of logfiles and of video or audio recordings. Many methodological frameworks have been proposed for further analysis of these data. However there seems to be a lack of widely available tools to support this process. In this frame we developed tools to support analysis and interpretation of field data in studies of collaborative problem solving. In the rest of the paper, we provide a brief introduction to these tools and their typical use. 2.1. Playback and annotation of event logfiles The first use of the analysis tools is related to the off-line presentation and annotation of logfiles, which have been produced during learning activities. These logfiles contain time-stamped actions and text messages of the partners engaged in problem solving, in sequential order, in the form