Continuous-time and discrete-time Markov chains often face two major challenges, state space explosion and uncertainty in transition rates or probabilities. State space explosion can be mitigated through state aggregation, which reduces the number of states. However, aggregation typically introduces uncertainty in the transition matrix, resulting in a Markov process with imprecise parameters. This paper defines several variants of parameter uncertainty and presents methods to compute bounds for stationary, transient, and accumulated rewards in Markov chains with uncertain transition matrices and reward structures. For certain types of parameter uncertainty, the problem can be reformulated as an optimization problem for Markov decision processes, allowing the computation of tight bounds. For other variants, however, the resulting optimization problems are non-convex, and tightness of the computed bounds cannot, in general, be guaranteed.

In [1] a new traffic shaping scheme denoted as Quasi-Deterministic Transmission Policy’ (QDTP) has been proposed and analyzed under deterministic and stochastic assumptions. This model is extended by considering phase type distributed and possible correlated arrivals, delays and services combined with load dependent behavior. Load dependent behavior is important to model overload situations at processing devices in a realistic way. It is shown that the resulting model describes a Quasi Birth Death Process (QBD) without state dependent rates for the delays and a Level Dependent Quasi Birth Death Process (LDQBD) with state dependent rates for the delays. We present an approach based on the recursive computation of QBDs or LDQBDs which can be applied if the number of states remains moderate. Larger models can be solved by a block iteration method. Finally, we show how the population in the system can be minimized by choosing an appropriate delay rate or vector of delay rates.

Leaky buckets are commonly used for access control in networks, where access control stands for traffic regulation at the ingress of the network. In network calculus, which is often applied for performance analysis or dimensioning of networks, leaky buckets are the model behind piecewise linear arrival curves that specify an input bound to a network. In this paper we present the analysis of leaky bucket based access control under stochastic arrivals using fluid queues, when the access control is implemented by possibly more than one leaky buckets. This results in methods to dimension parameters of access control for different stochastic arrival processes including correlated arrivals. The approach is one step to bridge the gap between classical stochastic analysis using queuing networks and deterministic analysis using network calculus. Results are presented for stochastic arrival processes using numerical methods and for measured arrivals using trace driven simulation.

In production scenarios, uncertainty in production times, and scrap rates is common. Uncertainty can be described by stochastic models that need continuous updates due to changing conditions. This paper models probability distributions and estimates their parameters from real-world data on processing times and scrap rates. It uses live fitting to identify timely changes in the data sets, comparing different distributions. The fitting and live fitting approaches are applied to data from a real production system to compare the goodness of fit for different distributions and to demonstrate the reliability of the reaction to changes in the input data. Data and probability estimations are categorised, and a concept combining simulation and optimisation models is developed to optimise scheduling. The vision of the research presented in this paper is an online approach that determines quasi-optimal schedules for production systems based on current data from the system and its environment.

We consider Markov fluid models with two infinite buffers, whose fluid rates ensure that the fluid level of buffer 1 is never larger than the one of buffer 2. For the model, we derive the system of PDEs describing the transient behavior and expressions for the stationary analysis in the Laplace domain. Examples for the application of the model in the context of network access control are presented. Numerical results are compared with detailed simulation results to show that the numerical approach computes the required measures with sufficient accuracy.

We study the relation of Markov fluid queues and QBD processes in this paper. Ahn and Ramaswami presented results about this relation and provided a stochastic interpretation based reasoning in [1]. In the current work, first we provide an algebraic proof for that relation. After that, we present a negative result about the potential extension of the QBD based analysis Markov fluid queues to Markov fluid queues with two buffers. We present a 2-dimensional QBD process, which could be a candidate for describing the stationary behaviour of the related Markov fluid queue, but it turns out that the QBD based behaviour is different from the one of the Markov fluid queue.

Analysis of trace data and modeling of data by appropriate distributions are essential for constructing accurate stochastic models. However, software support for these tasks is fragmented across various tools and libraries, including but not limited to tools like ExpertFit, MATLAB, R or software libraries like PyStats. This dispersion of software components can be challenging for users who lack experience with these software packages and requires, even for experienced users, a huge effort to configure adequate tool chains for data analysis and modeling. This paper introduces the first version of the ASMo tool, designed to collect various techniques for data analysis and modeling under a user-friendly umbrella. ASMo consolidates different data inspection and fitting approaches into a single platform, simplifying the process for modelers. It offers support for data inspection, analysis, calculation, and visualization of various statistical measures. Additionally, ASMo provides fitting algorithms for standard, mixture, and phase-type distributions and can directly export random variate generators for the fitted distributions. ASMo is intended as an open environment which can be easily extended by integrating new methods.

Single buffer Markov fluid models are well understood in the literature, but the extension of those results for multiple buffers is still an open research problem. In this paper we consider one of the simplest Markov fluid models (MFM) with 2 buffers of infinite capacity, where the fluid rates ensure that the fluid level of buffer 1 is never larger than fluid level of buffer 2. In spite of these restrictions, the stationary analysis is non straightforward with the available analysis tools. We provide an analysis approach based on the embedded time points at the busy-idle cycles of buffer 1.

Stochastic graph models and algorithms for the computation of different stochastic shortest path problems (SSPPs) continue to play an important role in various fields of computer science and operations research, with recent applications ranging from vehicle routing to social networks and decision problems in finance. In this paper, we first introduce stochastic graph models where weights to pass an edge in a graph are, possibly correlated, random variables. Depending on the application context and the interpretation of uncertain edge weights, random variables either model the time to pass an edge or describe a reward one gains by traversing an edge. For modeling the random variables discrete phase type distributions are used. Several algorithms for the computation of optimal policies which depend on the current weight upon arriving at a vertex while still exploiting weight dependencies among edges are developed. It will be shown that with correlated edge weights the stochastic shortest path problems become PSPACE-complete and the solution of instances with a bounded number of phases is NP-hard such that for an exact computation of the shortest path, algorithms from partially observable Markov decision processes (POMDPs) or mixed integer linear programming (MILP) have to be applied. Alternatively, an efficient heuristic algorithm can be used which often yields good or even optimal results with a small effort.

Planning of missions by fully decentralized and autonomous robot swarms in dynamically changing disaster areas is a tough challenge. Therefore, this paper provides a novel real time autonomy core for unmanned ground vehicles (UGVs) that generates usable, but not necessarily optimal trajectories based on specific missions. These are defined implicitly, without assigning goal states to single agents, by direct potential functions which may include nonlinearities, unsteadiness, and discrete parts for specifying complex behavior which is achieved without state machines or behavior trees. In addition, the autonomy core is designed to exchange small data packages between agents via unreliable mesh networks based on UDP or LongRange (LoRa) or a physically simulated mesh network inside the simulation.The availability of information from other agents improves the behavior of the swarm but is not required to fulfill a mission. Interfaces to common robotic middleware are provided to integrate simulations and real agents.

Planning missions by truly autonomous robots is a challenge. This paper presents a novel approach to design mission functions for optimization-based controllers that generate trajectories without explicit goal specifications for each robot. Potential fields are used to implicitly describe the goal of a mission. This allows one to model a great variety of missions including nonlinearities, discontinuities, and discrete parts. The proposed control algorithm is designed for swarms in which the communication is based on unreliable mesh networks requiring a completely decentralized control. The selection of the missions presented in this paper is mostly requested for disaster areas, defense and security operations, and logistics. Furthermore, experiments express the functionality of the chosen mission functions and the performance of the entire approach.

Markov Reward Models are widely used for performance or dependability analysis. Usually parameters of these models are assumed to be exactly known, results can then be computed numerically from the underlying Markov Reward Process. However, most times model parameters are estimated and are thus subject to uncertainty such that analysis becomes challenging and is usually done by analyzing the model for a large sample of possible input parameter settings which requires a huge effort if several parameters are uncertain. An alternative to this brute force approach is the computation of a simpler surrogate model from a smaller set of samples and apply the surrogate as a substitute for the Markov Reward Model. This paper presents several surrogate models, shows how they can be applied for Markov Reward Models and compares their quality by means of three different examples.

Dynamic Fault Trees (DFTs) are a powerful and widely used class of models for reliability analysis of technical systems. They describe the relation between failure times of elementary components and failures of the system modeled by the DFT. Failure times of elementary components are assumed to be independent and often exponentially distributed. Then the underlying stochastic process is a Continuous Time Markov Chain (CTMC) which is often analyzed numerically. In this paper, we use phase type distributions to model failure times of elementary components and extend DFTs by introducing two new types of nodes to express different variants of correlation between failure times which often can be observed in real systems. Since the use of phase type distributions enlarges the state space of the CTMC, compositional techniques allowing a compact representation of the generator matrix and analysis techniques exploiting this compact representation are also introduced. In particular, analysis techniques are presented that exploit the specific structure of the DFT.

The paper presents a new approach and a related software environment for the parallel simulation of swarms of autonomous robots in real time. The software environment has been developed for model based analysis of algorithms to control large swarms of distributed autonomous mobile robots communicating over an unreliable and capacity restricted wireless network. It includes a physical simulation of static obstacles, dynamic obstacles with scriptable movement, soil condition, active jammers, static and dynamic link obstacles with configurable damping as well as noise floors. The simulated ground based mobile robots use control particle belief propagation (C-PBP) as a randomized and sample based model predictive closed loop controller in combination with cost functions to evaluate the situations. We emphasize where the use of shared memory parallelism is beneficial and which inaccuracies in computations are acceptable to increase performance without losing realism.

The paper presents a new approach and a related real-time parallel simulation tool for modeling and analyzing a swarm of more than 60 distributed autonomous mobile robots communicating over an unreliable and capacity restricted wireless communication network. It includes a physical simulation of static obstacles, dynamic obstacles with scriptable movement, soil condition, active jammers, static and dynamic link obstacles with configurable damping as well as thermal noise. The simulated ground based mobile robots use CPBP [12] as probabilistic model predictive closed loop controller in combination with gradient free cost functions to evaluate complex unsteady goodness aggregations. The goal of this approach is the development of connection aware swarm behavior for complex missions such as terrain exploration, formations, convoy escorting or creation of a mobile ad hoc network in dynamic disaster areas under realistic environmental conditions. The missions can be combined in any manner and the target extraction of the high-level commands for each agent is done implicitly. To validate the developed behavior, the independent software control kernel is able to be used on real robots as well.

In this paper we present results for bivariate exponential distributions which are represented by phase type distributions. The paper extends results from previous publications [3, 11] on this topic by introducing new representations that require a smaller number of phases to reach some correlation coefficient and introduces different ways to describe correlation between exponentially distributed random variables.

In production systems, scheduling problems need to be solved under complex environmental conditions. In this paper, we present a comprehensive scheduling approach that is applicable in real industrial environments. To cope with the parameter uncertainty of real world problems, forecasting, classification, and simulation techniques are combined with heuristic optimization algorithms. Thus, the approach allows for identifying and including demand fluctuations and scrap rates, and offers a selection of suitable schedules depending on particular demand constellations in scheduling. Furthermore, we adapt seven optimization algorithms for two-stage hybrid flow shops with unrelated machines, machine qualifications, and skipping stages with the objective to minimize the makespan. The combination of methods is validated on a real production case of the automobile industry. The paper shows for the application case that metaheuristics provide significantly better results than SPT and safety factors, above a certain size, can reduce their effect preventing incomplete demand positions.

Phase Type Distributions (PHDs) and Markovian Arrival Processes (MAPs) are established models in computational probability to describe random processes in stochastic models. In this paper we extend MAPs to Multi-Dimensional MAPs (MDMAPs) which are a model for random vectors that may be correlated in different dimensions. The computation of different quantities like joint moments or conditional densities is introduced and a first approach to compute parameters with respect to measured data is presented.

In modern production systems, scheduling problems have to be solved in consideration of frequently changing demands and varying production parameters. This paper presents a approach combining forecasting and classification techniques to predict uncertainty from demands, and production data with heuristics, metaheuristics, and discrete event simulation for obtaining machine schedules. The problem is a hybrid flow shop with two stages, machine qualifications, skipping stages, and uncertainty in demands. The objective is to minimize the makespan. First, based on the available data of past orders, jobs that are prone to fluctuations just before or during the production phase are identified by clustering algorithms, and production volumes are adjusted accordingly. Furthermore, the distribution of scrap rates is estimated, and the quantiles of the resulting distribution are used to increase corresponding production volumes to prevent costly rescheduling resulting from unfulfilled demands. Second, Shortest Processing Time (SPT), tabu search, and local search algorithms are developed and applied. Third, the best performing schedules are evaluated and selected using a detailed simulation model. The proposed approach is validated on a real-world production case. The results show that the price for a very robust schedule that avoids underproduction with a high probability can significantly increase the makespan.