Sensor data streaming platforms feed pervasive applications with data through continuous processing tasks. Outliers should be constantly removed from the data. The existence of ontologies and their semantics to express the different properties of sensing devices in semantic models can be leveraged towards solving the problem of outlier detection. We present an outlier detection data stream operator based on sensor ontology-based definitions. This outlier detection operator could be defined and adjusted to fit in different use cases with different requirements. We provide a proof of concept implementation of the operator using a Data Stream Management System and evaluate the effect of increasing outlier rates in data sets on the overall performance of the outlier detection.
The integration of machine learning (ML) approaches in sensor-based applications in the field of pervasive computing is becoming increasingly prominent due to the increasing number of sensor-based applications in general and the continuous adaption of ML approaches to new domains. Several ML models are used within a processing pipeline that operates on the same sensor data. Still, the cloud computing approach is a straightforward solution where all sensor data is sent to the cloud before processing, which is inefficient according to resource utilization. Appropriate management of the different processing tasks for ML models enhances resource utilization. This paper proposes an architecture for resource-aware classification empowered by an ML model management (MLMM) framework and a distributed data stream management system (DDSMS). First, the classification pipeline is decomposed and implemented as data stream operators. Second, ML models are retrieved from an MLMM framework considering preprocessing, segmentation, and feature alignment to enable an effective redundancy elimination. Finally, the classification pipeline is deployed using resource-aware operator placement optimization. The evaluation results on a real-world scenario of a sensor-based activity classification pipeline for dairy cows show that our approach can reduce network utilization up to 98.9%.
The increasing availability of indoor and outdoor location sensors raises the interest for understanding the mobility in different places, e.g., in museums, train stations, or cities. One of the highly used approaches for understanding the mobility of individuals is to detect staypoints from trajectories. Although offline detection of staypoints fits best for long-term planning, some applications require online staypoint detection from trajectories, like online recommendation systems. Latency plays an important role here. In this paper, we propose a stream-based approach for staypoint detection, which can be realized by applying a Data Stream Management System. We claim that the proposed approach can detect staypoints with low latency from the high-frequency location update streams. To evaluate our approach, we compare it with a batch-based approach on real data from an indoor tracking system and Geolife dataset. The results demonstrate that the online approach detects staypoints with much lower latency compared to traditional approaches like offline and batch processing. Moreover, we prove that the accuracy of the stream-based approach is similar to the batch-based approach.
The energy grid is changing rapidly to include volatile, renewable energy sources to help achieve climate goals. The transition to a smart grid, including smart meters for the metering and communication of the energy consumption, helps with that transition. The smart meters provide a stream of measurements, which can be used for additional services, such as visualization of power consumption. Detecting switching events, when devices in a household are switched on or off, is one possible application on smart meter data. The goal of the ACM DEBS Grand Challenge 2020 is to implement a live switch detection on a data stream from a smart meter for Non-Intrusive Load Monitoring. This paper presents a solution for the challenge with a general purpose and open source data stream management system that focuses on reusable, generic operators instead of a custom black-box implementation.
Spatio-temporal data streams from moving objects have become ubiquitous in the recent years, not only, but also in the maritime domain. The Automatic Identification System (AIS) is an important technology in the maritime domain that creates huge amounts of streaming moving object data and enables new use cases. The data streams can improve the situation awareness, help Vessel Traffic Services (VTSs) to get an overview of certain situations and detect upcoming critical situations automatically. For these use cases, queries have to be processed on data streams with continuous results and little delay. To reach this goal, Data Stream Management Systems (DSMSs) lay a foundation to process data streams, but lack the capabilities for spatio-temporal query processing. We tackle this research gap with techniques known from moving object databases and integrate those into the stream processing. We present a system that integrates the moving object algebra from moving object databases into the interval approach from data stream processing to run queries on AIS data. This new approach allows us to define very diverse spatio-temporal queries on AIS data streams, such as radius queries, k-nearest neighbors (kNN) queries as well as queries with moving polygons. Additionally, the approach allows us to use short-time prediction to detect situations before they occur, e. g., to avoid collisions. Our results show that the system is very flexible, offers a clear semantics and produces results on AIS streams with many vessels with low latency.
Positional data from small and mobile Global Positioning Systems has become ubiquitous and allows for many new applications such as road traffic or vessel monitoring as well as location-based services. To make these applications possible, for which information on location is more important than ever, streaming spatial data needs to be managed, mined, and used intelligently. This article provides an overview of previous work in this evolving research field and discusses different applications as well as common problems and solutions. The conclusion indicates promising directions for future research.
Spatio-temporal data from Moving Objects is often available as a live data stream and needs to be processed accordingly. The trajectories update while the objects are moving. Even though the data arrives in a streaming manner, significant delays between location updates are possible, resulting in delayed or less accurate results of continuous queries on the trajectories. That can be an important issue when using the queries for real-world decisions, e. g., with Automatic Identification System (AIS) data for maritime navigation. Additionally, short-time predictions can be useful to get early warnings for critical situations. Previous work does not cover this problem for streaming applications, as existing systems are mainly Moving Object Databases, which are not optimized for streaming data. In this work, we describe how spatio-temporal inter and extrapolation can be integrated into Data Stream Management Systems and which challenges have to be solved doing so.
The complexity of critical systems such as traffic management has dramatically increased over the last decades, since they involve more and more sensors to derive distributed situational awareness. Most existing systems require an a-priori configuration of sensors or need human intervention to adapt to changes. Furthermore, analysis of data quality or query plan reliability is often not possible and management of recorded data is done by hand. Our goal is to support the research, development, evaluation and demonstration of such systems. Within this study we analyze requirements and challenges for the data management of sensor based research environments and present a data stream based architecture which fulfills these requirements.
Machine Condition Monitoring (MCM) is an important topic for the reliability of industrial machines in increasingly interconnected production facilities. The analysis of a huge amount of data to get information about the machine's condition is a difficult challenge. Current solutions for these analyses are often very specific, need a lot of manual configuration or are difficult to apply. In this paper, we present a system that uses anomaly detection in data streams to find hints for faulty machines in the data. The basis of this system is a Data stream management system (DSMS), which can handle huge amounts of streaming data and simplifies the definition of analyses. Due to the anomaly detection algorithms, the approach can be applied to a variety of data and scenarios. The outcome is a system that allows live analysis of machine data for MCM.
Conventional database management systems are usually not capable to deal with continuous processing of potentially infinite data streams. Therefore, special data stream management systems and frameworks are developed. They use continuous queries which produce also streams as results, so that static visualization is not feasible, since the results are changing constantly. To handle this, we developed a dashboard concept, which we want to propose in this demonstration. A dashboard can be considered as an control or monitoring panel for real time data stream results. The user is free to define and configure individual dashboard parts. Each part is connected to a (user defined) continuous query, whose results are received and visualized in real-time. In this demonstration, we provide different data stream sources, continuous queries and dashboard parts. With them, the user can compose his own individual presentation of his processing results.
In this paper, we provide a solution for the ACM DEBS Grand Challenge 2015 (GC 2015) that deals with the analysis of taxi trips in New York based on the data stream management framework Odysseus [1].
The development of algorithms for online Collaborative Filtering (CF) in the past few years enables to add new rating data to existing models. The Recommender System (RecSys) task changes from calculating recommendations from a static and finite dataset to continuously processing rating data. Instead of using stream processing frameworks to implement CF algorithms, we present a prototype that extends the open source Data Stream Management System (DSMS) Odysseus in a generic and domain-independent way. The user can build a custom RecSys that benefits from existing DSMS features by defining a continuous query with a declarative query language.
In dieser Arbeit wird eine exible, erweiterbare und dom an-enunbh angige Integration von Online-RecSys-Funktionen inein Datenstrommanagementsystem (DSMS) vorgestellt. Da-zu werden neue logische Operatoren eingef uhrt, mit der Nut-zer eines DSMS auf einer abstrakten Ebene RecSys-Funk-tionen nutzen k onnen. Des Weiteren wird eine beispielhafteRealisierung durch physische Operatoren vorgestellt, wie sieaus den logischen Operatoren durch Transformationsregelnerzeugt werden kann. Durch dieses Konzept k onnen Benut-zer ein RecSys mit Hilfe einer Anfragesprache auf die do-m anenspezi schen Bed urfnisse anpassen und mit anderenFunktionen eines DSMS kombinieren. Des Weiteren bringtein DSMS einige Eigenschaften (z.B. Anfrageplanoptimie-rung, Fragmentierung, Scheduling etc.) mit, von denen einOnline-RecSys pro tieren kann. Die Flexibilit at eines DSMSerm oglicht den Vergleich und die Evaluation verschiedenerRecSys-Algorithmen durch den Benutzer.
Recommender-Systeme (RecSys) findet man in vielen Informationssystemen. Das Ziel eines RecSys ist das Interesse eines Benutzers an bestimmten Objekten (engl. item) vorherzusagen, um aus einer grosen Menge an Objekten diejenigen dem Benutzer zu empfehlen, fur die das vorhergesagte Interesse des Benutzers am grosten ist. Die zu empfehlenden Objekte konnen zum Beispiel Produkte, Filme/Videos, Musikstucke, Dokumente, Point of Interests etc. sein. Das Interesse eines Benutzers an einem Objekt wird durch eine Bewertung (engl. rating) quantifiziert. Die Bewertung kann explizit durch den Benutzer angegeben (der Benutzer wird dazu aufgefordert, ein bestimmtes Objekt zu bewerten) oder implizit vom Verhalten des Benutzers abgeleitet werden (im einfachsten Fall durch eine binare Bewertung: Objekt genutzt vs. nicht genutzt). Mit Methoden des maschinellen Lernens wird aus bekannten Bewertungen ein Modell trainiert, dass unbekannte Bewertungen vorhersagen kann. Fur die Bestimmung der Empfehlungsmenge werden die Bewertungen aller unbewerteten Objekte fur einen Benutzer vorhergesagt und die best-bewerteten Objekte empfohlen.
Überlast bedeutet, dass an ein System mehr Anforderungen gestellt werden, als es erfüllen kann. Im schlechtesten Fall ist es nicht mehr ansprechbar oder stürzt ab. Für Datenbankmanagementsysteme (DBMS) existiert eine spezielle Komponente, Admission Control (AC) genannt, welche die Systemlast überwacht, eintreffende Anfragen vor deren Ausführung überprüft und ggfs. zurückstellt. Für kontinuierliche Anfragen, welche permanent ausgeführt werden, reicht diese Art von AC jedoch nicht aus: Es kann zur Konfliktlösung nicht mehr auf die Terminierung einer Anfrage gewartet werden. Zudem ist die Verarbeitung datengetrieben, d. h. Umfang und Inhalt der Daten können stark variieren. Diese Arbeit stellt ein Konzept vor, wie eine flexible und anpassbare AC-Komponente für kontinuierliche Anfragen auf Basis eines Event-Condition-Action-Modells (ECA) umgesetzt werden kann. Zur Regeldefinition wird die einfache Sprache CADL vorgestellt, die eine hohe Flexibilität und Anpassbarkeit der kontinuierlichen AC an konkrete Systeme und Hardware erlaubt. Die Evaluation zeigt mittels Odysseus, einem Framework für Datenstrommanagementsysteme, dass das Konzept effizient funktioniert und effektiv zur Lastkontrolle und Ergreifung von Maßnahmen zur Lastreduktion eingesetzt werden kann.
Smart homes are equipped with multiple sensors and actuators to observe the residents and environmental phenomena, to interpret the situation out of that, and finally, to react accordingly. While the data processing for a single smart home is facile, the data processing for multiple smart homes in one smart building is more complex because there are different people (e.g., like several residents, administrators, or a property management) with different interests concerning the processed data. On that point, this chapter shows which kind of typical roles can be found in a smart building and what requirements and challenges they demand for managing and processing the data. Secondly, Data Stream Management Systems (DSMS) are introduced as an approach for processing and managing data in a smart building by presenting an appropriate architecture. Finally, the chapter discusses further concepts from DSMS and illustrates how they additionally meet and solve the requirements and the challenges.
Die Abteilung Informationssysteme (IS) gehort zum Department fur Informatik der Fakultat fur Informatik, Wirtschaftsund Rechtswissenschaften an der Carl-vonOssietzky-Universitat Oldenburg. IS ist mit der Grundung im Jahr 1987 aufgrund der Berufung von Prof. H.-JurgenAppelrath eine der beiden altesten, die bzgl. Mitarbeiterzahl groste und bzgl. ihrer Profilbildung dynamischste der aktuell insgesamt 20 Abteilungen im Department fur Informatik, weil die in ihr erforschten, entwickelten und in Anwendungen evaluierten Technologien uber gut 25 Jahre ungewohnlich stark wechselten (Abb. 1). Die sich naturlich zeitlich uberlappenden Phasen der Entwicklung von IS lassen sich wie folgt charakterisieren: Von 1987 bis Anfang der 90er Jahre standen auf Basis der von Lehrstuhlinhaber und Mitarbeitern von der ETH Zurich mitgebrachten Vorarbeiten die Themen Planungssysteme und Multimedia, aber dann auch neu User-InterfaceManagement-Systeme im Vordergrund. Diese Themen wurden in den 90er Jahren erganzt und spater abgelost durch aktive und temporale Datenbanksysteme, Data Warehouses incl. Technologien zu Datenanalyse, Datenqualitatsmanagement und Datenschutz, hier zunehmend mit Anwendungen aus Medizin und Gesundheitswesen wie z. B. in bevolkerungsbezogenen Krebsregistern [1]. Ende der 90er Jahre verstarkten sich Arbeiten zu digitalen Bibliotheken, virtuellen Laboren und generell multimedialem E-Learning, wahrend parallele