This paper reports on a study of the design, development and evaluation of two teaching analytics dashboards that visualize students' repertory grid exercise data. The technical objective of the dashboards is to support teachers to investigate and compare personal constructs and element ratings by students for given topics of study at the individual student, group or classroom levels of analysis. The pedagogical objective of the dashboards is to facilitate formative assessment feedback to students and feed-forward refinements to ongoing instructional practices. The teaching analytics dashboards were evaluated with six university teachers in a bio-metric usability study with integrated physiological and performance measures. Eye-tracking data from the two dashboards showed the relative importance of dashboard areas in each case in terms of aggregate gaze allocation. Findings informed the iterative design of the dashboards leading to teachers not being adversely affected by the visual clutter of the less important dashboard areas in their pedagogical decision-making. The results also showed that the dashboards were efficient and effective from a task performance perspective and were rated to be pleasant from a subjective satisfaction perspective. Regarding education technology innovation, teachers reported that such dashboards are lacking in their regular practice and would recommend their use in formal business educational settings.

A health record database contains structured data fields that identify the patient, such as patient ID, patient name, e-mail and phone number. These data are fairly easy to de-identify, that is, replace with other identifiers. However, these data also occur in fields with doctors’ free-text notes written in an abbreviated style that cannot be analyzed grammatically. If we replace a word that looks like a name, but isn’t, we degrade readability and medical correctness. If we fail to replace it when we should, we degrade confidentiality. We de-identified an existing Danish electronic health record database, ending up with 323,122 patient health records. We had to invent many methods for de-identifying potential identifiers in the free-text notes. The de-identified health records should be used with caution for statistical purposes because we removed health records that were so special that they couldn’t be de-identified. Furthermore, we distorted geography by replacing zip codes with random zip codes.

. In this article the authors investigated a visualization tool (uVis) for end-user developers, in order to see how end users actually use it. The tool was an early version and the investigation helped the authors to improve it. The investigation showed that users appreciated the simple formula language, the coordinated panels, and the drag-and-drop mechanism. However, the most important thing for them was the immediate response when they changed something, for instance part of a formula. The entire visualization was updated immediately without having to switch from development view to production view. With uVis, developers construct a visualization from simple visual components such as boxes, curvePoints, and textboxes. All component properties such as Top and BackColor can be complex formulas similar to spreadsheet formulas. The operands in the formula can address relational data in a database, other visual objects, and dialog data provided by the user. A special Rows property can bind to a database query and make the component replicate itself for each row in the query. In this way, traditional as well as novel visualizations can be constructed. The most serious usability problems were data binding and not noticing errors (errors were shown in an error list, but not in the formula that had the error). There were many other usability problems. Removing them would speed up learning and make the tool more successful.

This paper presents the concept of iKlassroom as a real-time and realplace teaching analytics application. We present design sketches and screenshots, outline a usage scenario for classroom practice and identify potential contributions to learning sciences research.

Creating and customizing visualization for electronic health record data requires a close collaboration with clinicians, to understand their tasks, needs and mental model. This process can develop into an infinite process. Taking into consideration the existence of clinicians with advanced IT knowledge, but not programmers, we focus on engaging them to create their own visualizations. This paper presents how clinicians can use uVis Studio to create three visualizations by dragging and dropping controls into the design panel, and specifying formulas for each control in the property grid.

Existing custom visualization tools provide cognitive artefacts that can reduce the cognitive barriers designers encounter in visual mapping, mapping data to visual primitives (e.g. ellipse, rectangle) and their properties (e.g. colour, size.) However, our experience with designers shows that the existing artefacts are insufficient. In particular, designers struggle to understand the relationship between visual primitives, their properties, and data. Moreover, it is hard to verify the correctness of the visual mappings. In response, we developed a new cognitive artefact, the inspector, that explicitly shows the data behind visual primitive and their properties. We evaluated the impact of the inspector on usability with seven designers. The result shows that the inspector reduced usability problems and helped designers verify the visual mappings.

Repertory grid tools systematically collect data that consists of a topic, its elements, constructs and element ratings. Gaining meaningful insights from repertory grids data is a challenge because data analysis is timeconsuming and a significant mental effort is needed. Visualizations aim at facilitating data analysis through a visual and interactive approach, which allows users to understand their data, reflect, and make better decisions. This paper presents an interactive visualization tool for teachers and students. The tool visualizes repertory grid data using two dashboards, where teachers and students can investigate constructs and rating elements of students at the individual or group level. Visualizing the repertory grid data is an initial attempt towards teaching analytics. Future work will focus on evaluating the tool in a real setting with teachers and students, and collecting suggestions for improvement.

A toolkit facilitates the visualization development process. The process can be further enhanced by integrating the toolkits in development environments. This paper describes how the uVis toolkit, a formula-based visualiza- tion toolkit, has been extended with a development environment, called uVis Studio. Instead of programming, developers apply a Drag-Drop-Set-View-Interact approach. Developers bind controls to data, and the Studio gives immediate visual feedback in the Design Panel. This is a novel feature, called What-You-Bind-Is-What- You-Get. The Studio also provides Modes that allow developers to interact and view the visualization from the end-user's perspective without switching workspace, and Auto-Completion; a feature of the Property Grid that provides suggestions not only for the formula language syntax but also for the tables, the table elds and the relationships in the database. We conducted a usability study with six developers to evaluate if the Studio and its features enhance cognition and facilitate the visualization development. The results show that developers appreciated the Drag-Drop-Set- View-Interact approach, the What-You-Bind-Is-What-You-Get, the Auto-Completion and the Modes. Several usability problems were identified, and some suggestions for improvement include: new panels, better presentation of the Modes, and better error messages.

This paper investigates End-User Development of Information Visualization. More specifically, we investigated how existing visualization tools allow end-user developers to construct visualizations. End-user developers have some developing or scripting skills to perform relatively advanced tasks such as data manipulation, but no formal training in programming. 18 visualization tools were surveyed from an end-user developer perspective. The results of this survey study show that end-user developers need better tools to create and modify custom visualizations. A closer collaboration between End-User Development and Information Visualization researchers could contribute towards the development of better tools to support custom visualizations. In addition, as empirical evaluations of these tools are lacking both research communities should focus more on this aspect. The study serves as a starting point towards the engagement of end-user developers in visualization development.

Popular tools for constructing user interfaces use the drag-drop-setproperty principle. The developer drops visual components (boxes, labels, etc.) on the screen and defines their properties, e.g. position, color and text. However, data presentation and user interaction are very limited. To exceed this limit, programming is needed. Many end-user developers are familiar with the popular tools, but are uncomfortable with programming. Could we improve the drag-drop-set-property principle so that they can make non-standard data visualization and interaction without programming? This paper presents a tool (uVis) that takes a long step in this direction. The principle is to allow each property to be a spreadsheet-like formula that computes position, color, etc. A formula can combine data from several database tables with data about components and dialog data provided by the end-user. Formulas can also handle events and provide interaction.

Multi-step processes are common in healthcare, software development, and other dynamic environments. Managers who monitor processes are overwhelmed by information, and often limited in the time to investigate data from different viewpoints. We propose a visual approach to facilitate retrospective analysis, to promote ∗e-mail: kopa@itu.dk †This work was done while visiting the University of Maryland, College Park, Maryland, USA ‡e-mail:sureyya@cs.umd.edu §e-mail:plaisant@cs.umd.edu ¶e-mail:ben@cs.umd.edu timely completion of multi-step processes, and improve performance. It uses a novel visualization called the Process Completion Diagram (PCD) that aggregates event-logs into in-time, late and not-completed completions, and visualizes those using shapes and colors. The interactive Multi-Step Process Visualization (MSProVis) combines a number of PCDs and allows users to review and compare series of PCDs at three levels of detail, allowing comparisons between steps or between actors executing those steps. MSProVis computes default thresholds that define in-time and late completions. These might trigger alarms, and allow managers to adjust those thresholds interactively. A pilot usability test with five participants showed that all participants could learn how to read almost all the information presented in the PCD without training in less than five minutes, and use MSProVis effectively. They carried out nine tasks successfully, which lasted on average seven minutes. Suggestions for improvement include color usage, and label and shape positioning.

Many visualization tools allow the implementation of custom (non-standard) visualizations, but they differ in approach. The approaches vary from imperative to declarative programming. Moreover, some tools provide environments that assist designers in implementing visualizations. Which approach supports designers best in implementing custom visualizations? What is lacking? To answer these questions, we compared the approaches of four recent visualization tools that support custom visualizations using an example. Further, we evaluated the approaches using the framework of the Cognitive Dimensions of Notations (CDs). Our findings favour notations that use declarative rather than imperative programming, and environments that allow exploration rather than dialogue-dependant ones.

Several information visualization toolkits and tools have been developed to facilitate users work. However, evaluation studies for these toolkits and tools from a user perspective have been overlooked. In this paper, we evaluate 13 information visualization toolkits and tools developed in the last 10 years from a user perspective. For each user type (novice, savvy and expert), we investigated the toolkits and tools using 5 dimensions. This study shows that tools for savvy users—users with basic programming skills and domain knowledge— are lacking, and these users should be supported with more tools to construct advanced visualizations. We outline our approach to how savvy users can be engaged more in the development process and construct advanced visualizations. Information visualization community should focus more on moving the boundary of constructing advanced visualizations from expert users towards savvy users.

Popular tools for constructing user screens use the drag-drop-set-property principle. The developer drops components (buttons, text boxes, etc.) on the screen and defines their properties, e.g. position, color and text. Then the screen looks right, but it has little functionality. If you want real functionality or a custommade visualization, you have to switch to tools that are more like programming. Many domain experts are familiar with the popular tools and would like to provide functionality and visualize data, but they are uncomfortable with programming. Could we improve the drag-drop-set-property principle and cover their needs without asking them to program? This paper presents a tool (uVis) that takes a long step in this direction. The principle is to allow each property to be a formula that computes position, color, etc. A formula can combine data from several database tables with data about components and data entered by the enduser. Although aimed at non-programmers, the tool can also boost the performance of programmers. The paper presents the formula language, the tool and some user evaluations of the tool.

Electronic health records (EHR) contain a large amount of structured data and free text. Exploring and sharing clinical data can improve healthcare and facilitate the development of medical software. However, revealing confidential information is against ethical principles and laws. We de-identified a Danish EHR database with 437,164 patients. The goal was to generate a version with real medical records, but related to artificial persons. We developed a de-identification algorithm that uses lists of named entities, simple language analysis, and special rules. Our algorithm consists of 3 steps: collect lists of identifiers from the database and external resources, define a replacement for each identifier, and replace identifiers in structured data and free text. Some patient records could not be safely de-identified, so the de-identified database has 323,122 patient records with an acceptable degree of anonymity, readability and correctness (F-measure of 95%). The algorithm has to be adjusted for each culture, language and database.