The enormous growth of data in the last decades led to a wide variety of different database technologies. Nowadays, we are capable of storing vast amounts of structured and unstructured data. To address the challenge of exploring and making sense out of big data using visual analytics, the tight integration of such backend services is needed. In this article, we introduce BANKSAFE, which was built for the VAST Challenge 2012 and won the outstanding comprehensive submission award. BANKSAFE is based on modern database technologies and is capable of visually analyzing vast amounts of monitoring data and security-related datasets of large-scale computer networks. To better describe and demonstrate the visualizations, we utilize the Visual Analytics Science and Technology (VAST) Challenge 2012 as case study. Additionally, we discuss lessons learned during the design and development of BANKSAFE, which are also applicable to other visual analytics applications for big data.

Movement data sets collected using today's advanced tracking devices consist of complex trajectories in terms of length, shape, and number of recorded positions. Multiple additional attributes characterizing the movement and its environment are often also included making the level of complexity even higher. Simplification of trajectories can improve the visibility of relevant information by reducing less relevant details while maintaining important movement patterns. We propose a systematic stepwise methodology for simplifying and thematically enhancing trajectories in order to support their visual analysis. The methodology is applied iteratively and is composed of: (a) a simplification step applied to reduce the morphological complexity of the trajectories, (b) a thematic enhancement step which aims at accentuating patterns of movement, and (c) the representation and interactive exploration of the results in order to make interpretations of the findings and further refinement to the simplification and enhancement process. We illustrate our methodology through an analysis example of two different types of tracks, aircraft and pedestrian movement.

With its mission to move science into practice, the Visual Analytics Science and Technology Challenge has become an integrated part of the annual Visual Analytics Science and Technology Conference since its inception in 2006. In this article, we discuss how we can transfer this objective into a classroom setting by using the Visual Analytics Science and Technology Challenge datasets and by encouraging student submissions to the challenge. By means of Bloom's Taxonomy of Educational Objectives for Knowledge-Based Goals, we show how the Visual Analytics Science and Technology Challenge enables the integration of additional learning objectives into two types of courses: a dedicated course that focuses on the contest participation and an integrated course that uses the contest data to emphasize practical course elements. The core contribution of this article is that we assess the opportunities and pitfalls that we experienced at the University of Konstanz in Germany and Virginia Tech in the United States when augmenting the educational curriculum with the Visual Analytics Science and Technology Challenge.

We present the results of a controlled experiment to investigate the performance of different temporal glyph designs in a small multiple setting. Analyzing many time series at once is a common yet difficult task in many domains, for example in network monitoring. Several visualization techniques have, thus, been proposed in the literature. Among these, iconic displays or glyphs are an appropriate choice because of their expressiveness and effective use of screen space. Through a controlled experiment, we compare the performance of four glyphs that use different combinations of visual variables to encode two properties of temporal data: a) the position of a data point in time and b) the quantitative value of this data point. Our results show that depending on tasks and data density, the chosen glyphs performed differently. Line Glyphs are generally a good choice for peak and trend detection tasks but radial encodings are more effective for reading values at specific temporal locations. From our qualitative analysis we also contribute implications for designing temporal glyphs for small multiple settings.

We report the approach and results on the VAST 2012 MiniChallenge 2: Bank of Money Regional Office Network Operations Forensics. Using commercial data mining, visualization and database software such as KNIME, Tableau and MySQL as well as a custom-written source vs. destination IP pixel matrix, our team of students identified suspicious IRC traffic, an attack on the firewall, a drop in the firewall connections, an attempt for sensitive information exchange and a possible Distributed Denial-of-Service attack executed partly from a host within the bank network.

Real-time analysis of data streams has become an important factor for success in many domains such as server and system administration, news analysis and finance to name just a few. Introducing real-time visual analytics into such application areas promises a lot of benefits since the rate of new incoming information often exceeds human perceptual limits when displayed linearly in raw formats such as textual lines and automatic aggregation often hides important details. This paper presents a system to tackle some of the visualization challenges when analyzing such dynamic event data streams. In particular, we introduce the Event Visualizer, which is a loosely coupled modular system for collecting, processing, analyzing and visualizing dynamic real-time event data streams. Due to the variety of different analysis tasks the system provides an extensible framework with several interactive linked visualizations to focus on different aspects of the event data stream. Data streams with logging data from a computer network are used as a case study to demonstrate the advantages of visual exploration.

Colorpleth maps are commonly used to display election results, either by using one distinct color for representing the winning party in each district or by showing a proportion between two parties on a bi-polar colormap, for example, from red to blue representing Republicans vs. Democrats. Showing only the largest party may disable insights into the data whereas using bipolar colormaps works only reasonably well in cases of two parties. To overcome these limitations we introduce a new technique for visualizing proportions in such categorical data. In particular, we combine bipolar colormaps with an adapted double-rendering of polygons to simultaneously visually represent the first two categories and the spatial location. Our technique enables the recognition of close election results as well as clear majorities in a scalable manner. We proof our concept by applying our technique in a prototype implementation used to display election results from the U. S. Presidential election in 2008 and elections of the German Bundestag in 2005 and 2009. Different interesting findings are presented, which would not be recognizable when visualizing only the winner. As we additionally represent the party with the second most votes, we are able to show changes in the spatial distribution of the votes as well as outlier regions with exceptional results. Our visualization technique therefore enables valuable insights into categorical data with a spatial reference.

Routing in the Internet is vulnerable to attacks due to the insecure design of the border gateway protocol (BGP). One possible exploitation of this insecure design is the hijacking of IP blocks. Such hijacked IP blocks can then be used to conduct malicious activities from seemingly legitimate IP addresses. In this study we actively trace and monitor the routes to spam sources over several consecutive days after having received a spam message from such a source. However, the real challenge is to distinguish between legitimate routing changes and those ones that are related to systematic misuse in so-called spam campaigns. To combine the strengths of human judgement and computational efficiency, we thus present a novel visual analytics tool named Vistracer in this paper. This tool represents analysis results of our anomaly detection algorithms on large traceroute data sets with the help of several scalable representations to support the analyst to explore, identify and analyze suspicious events and their relations to malicious activities. In particular, pixel-based visualization techniques, novel glyph-based summary representations and a combination of temporal glyphs in a graph representation are used to give an overview of route changes to specific destinations over time. To evaluate our tool, real-world case studies demonstrate the usage of Vistracer in practice on large-scale data sets.

Firewalls have become essential components in the security concept of almost any modern computer network. Due to their relevance and central location in the network, their programming logic often survives several generations of administrators and hardware. Understanding the logic behind a firewall configuration is thus an important but challenging task for a network administrator. In general, there is a tendency to add new rules while old rules are only rarely changed or removed due to unexpected consequences in the network. In this paper we present a visualization tool to support the network administrator in this complex task of understanding firewall rule sets and object group definitions. The tool consists of a hierarchical sunburst visualization, which logically groups rules or object groups according to their common characteristics, a color-linked configuration editor and classical tree view components for rules and object groups. All these components are interactively linked to enable both exploratory and hypotheses testing tasks aimed at understanding the complex functionality of a firewall configuration. To verify our design, we present two case studies on the analysis of rule usage and on nested object groups and collected feedback from five firewall administrators.

The quality of a clustering not only depends on the chosen algorithm and its parameters, but also on the definition of the similarity of two respective objects in a dataset. Applications such as clustering of web documents is traditionally built either on textual similarity measures or on link information. Due to the incompatibility of these two information spaces, combining these two information sources in one distance measure is a challenging issue. In this paper, we thus propose a geodesic distance function that combines traditional similarity measures with link information. In particular, we test the effectiveness of geodesic distances as similarity measures under the space assumption of spherical geometry in a 0-sphere. Our proposed distance measure is thus a combination of the cosine distance of the term-document matrix and some curvature values in the geodesic distance formula. To estimate these curvature values, we calculate clustering coefficient values for every document from the link graph of the data set and increase their distinctiveness by means of a heuristic as these clustering coefficient values are rough estimates of the curvatures. To evaluate our work, we perform clustering tests with the k-means algorithm on a subset of the English Wikipedia hyperlinked data set with both traditional cosine distance and our proposed geodesic distance. Additionally, taking inspiration from the unified view of the performance functions of k-means and k-harmonic means, min and harmonic average of the cosine and geodesic distances are taken in order to construct alternate distance forms. The effectiveness of our approach is measured by computing micro-precision values of the clusters based on the provided categorical information of each article.

Online news sources produce thousands of news articles every day, reporting on local and global real-world events. New information quickly replaces the old, making it difficult for readers to put current events in the context of the past. The stories about these events have complex relationships and characteristics that are difficult to model: they can be weakly or strongly related or they can merge or split over time. In this article, we present a visual analytics system for temporal analysis of news stories in dynamic information streams, which combines interactive visualization and text mining techniques to facilitate the analysis of similar topics that split and merge over time. Text clustering algorithms extract stories from online news streams in consecutive time windows and identify similar stories from the past. The stories are displayed in a visualization, which (1) sorts the stories by minimizing clutter and overlap from edge crossings, (2) shows their temporal characteristics in different time frames with different levels of detail, and (3) allows incremental updates of the display without recalculating the past data. Stories can be interactively filtered by their duration and connectivity in order to be explored in full detail. To demonstrate the system's capabilities for detailed dynamic text stream exploration, we present a use case with real news data about the Arabic Uprising in 2011.

While in clear-cut situations automated analytical solution for data streams are already in place, only few visual approaches have been proposed in the literature for exploratory analysis tasks on dynamic information. However, due to the competitive or security-related advantages that real-time information gives in domains such as finance, business or networking, we are convinced that there is a need for exploratory visualization tools for data streams. Under the conditions that new events have higher relevance and that smooth transitions enable traceability of items, we propose a novel dynamic stream visualization called StreamSqueeze. In this technique the degree of interest of recent items is expressed through an increase in size and thus recent events can be shown with more details. The technique has two main benefits: First, the layout algorithm arranges items in several lists of various sizes and optimizes the positions within each list so that the transition of an item from one list to the other triggers least visual changes. Second, the animation scheme ensures that for 50 percent of the time an item has a static screen position where reading is most effective and then continuously shrinks and moves to the its next static position in the subsequent list. To demonstrate the capability of our technique, we apply it to large and high-frequency news and syslog streams and show how it maintains optimal stability of the layout under the conditions given above.

Modern communication infrastructures enable more and more information to be available in real-time. While this has proven to be useful for very targeted pieces of information, the human capability to process larger quantities of mostly textual information is definitely limited. Dynamic visual analytics has the potential to circumvent this real-time information overload by combining incremental analysis algorithms and visualizations to facilitate data stream analysis and provide situational awareness. In this book chapter we will thus define dynamic visual analytics, discuss its key requirements and present a pipeline focusing on the integration of human analysts in real-time applications. To validate this pipeline, we will demonstrate its applicability in a real-time monitoring scenario of server logs.

Treemaps are a powerful method to visualize especially time-invariant hierarchical data. Most attention is drawn to rectangular treemaps, because their space-filling layouts provide good scalability with respect to the amount of data that can be displayed. Since circular treemaps sacrifice the space-filling property and since higher level circles only approximately match the aggregated size of their descendants, they are rarely used in practice. However, for drawing circular glyphs their shape preserving property can outweigh these disadvantages and facilitate comparative tasks within and across hierarchy levels. The interactive ClockMap visualization effectively supports the user in exploring and finding patterns in hierarchical time-series data through drill-down, semantic zoom and details-on-demand. In this study, the technique’s applicability is demonstrated on a real-world dataset about network traffic of a large computer network and its advantages and disadvantages are discussed in the context of alternative layouts.

With the reliance of businesses, public institutions and individuals on large computer networks, maintaining their security becomes essential to ensure integrity. To achieve situational awareness, we developed Banksafe, which is a scalable, distributed and web-based visualization system to analyze health monitoring data and security datasets. To handle large amounts of data a cloud-based backend database is used to store and analyze the raw data. To evaluate the effectiveness of our approach we use both VAST 2012 mini challenges. Our case studies successfully identify suspicious events, trends and patterns using multiple visualizations.