This paper introduces a modular simulation testbed developed for decision support in logistics and urban planning within the tourism sector, specifically applied to & Scedil;anl & imath;urfa, Turkey, as part of the GeostaloVR project. The testbed integrates tourism value chains, urban development processes, and mobility flows to facilitate scenario testing and evaluation. By incorporating real-world data, the simulation framework enables the analysis of key performance indicators for sustainable urban tourism development. This approach aims to advance the application of modeling techniques to address complex challenges in urban tourism in order to provide valuable insights for stakeholders involved in urban and logistics planning.

Intralogistics systems, while complex, are crucial for a range of industries.One of their challenges is deadlock situations that can disrupt operations and decrease efficiency.This paper presents a four-stage framework for applying reinforcement learning algorithms to manage deadlocks in such systems.The stages include Problem Formulation, Model Selection, Algorithm Selection, and System Deployment.We carefully identify the problem, select an appropriate model to represent the system, choose a suitable reinforcement learning algorithm, and finally deploy the solution.Our approach provides a structured method to tackle deadlocks, improving system resilience and responsiveness.This comprehensive guide can serve researchers and practitioners alike, offering a new avenue for enhancing intralogistics performance.Future research can explore the framework's effectiveness and applicability across different systems.

In this paper, we explore the potential of multi-agent reinforcement learning (MARL) for managing the driving behavior of autonomous guided vehicles (AGVs) in production logistics environments with single-lane tracks, where deadlocks pose a significant challenge. We build upon previous work and adopt a MARL approach using the Proximal Policy Optimization (PPO) algorithm. We conduct a thorough hyperparameter search and investigate the impact of varying numbers of agents on the performance of the AGVs. Our results demonstrate the effectiveness of the MARL approach in addressing deadlocks and coordinating AGV behavior, as well as the scalability of the learned policy to different numbers of agents. The Bayesian optimization process and increased iteration count contribute to improved performance and more stable learning curves.

This study introduces an innovative strategy for addressing the Home Healthcare and Dial-a-Ride Problem (HHCDAP) concerning the transportation of medical staff and patients, taking into account the stochastic nature of service and travel times. The problem involves assigning suitable medical staff to patients and clients, determining the order of visits, and identifying opportunities for medical staff and patients to share trips. We propose two objective functions to minimize travel time for drivers and medical staff. This problem adheres to numerous constraints, including maximum work duration, maximum waiting time, professional qualifications, and vehicle capacity limitations. We test our approach on a small-scale instance to understand the trade-offs between minimizing drivers’ travel time and minimizing the travel and waiting times of medical staff and patients. Our results indicate that the proposed strategy enhances the efficiency of transporting medical staff and patients.

Decision-making in supply chains is challenged by high complexity, a combination of continuous and discrete processes, integrated and interdependent operations, dynamics, and adaptability. The rapidly increasing data availability, computing power and intelligent algorithms unveil new potentials in adaptive data-driven decision-making. Reinforcement Learning, a class of machine learning algorithms, is one of the data-driven methods. This semi-systematic literature review explores the current state of the art of reinforcement learning in supply chain management (SCM) and proposes a classification framework. The framework classifies academic papers based on supply chain drivers, algorithms, data sources, and industrial sectors. The conducted review revealed a few critical insights. First, the classic Q-learning algorithm is still the most popular one. Second, inventory management is the most common application of reinforcement learning in supply chains, as it is a pivotal element of supply chain synchronisation. Last, most reviewed papers address toy-like SCM problems driven by artificial data. Therefore, shifting to industry-scale problems will be a crucial challenge in the next years. If this shift is successful, the vision of data-driven decision-making in real-time could become a reality.

The planning of complex logistic systems must ensure collision- and deadlock-free operation of the logistic system. Problem-specific rule-based algorithms used so far are inflexible with respect to infrastructure changes and scale poorly with systems that grow larger. This paper shows a first approach to handle logistic deadlocks with machine learning. We present a conceptual approach on how to handle logistic deadlocks with artificial neural networks. The paper also provides a technical implementation with a single agent approach based on reinforcement learning with deep Q-networks. A discrete event simulation of an automated guided vehicle system is used as the learning environment. The first results show that artificial neural networks can learn to handle deadlock capable logistic systems with low complexity.

Discrete-event simulation is an established method to support decision making for planning tasks in production and logistics. However, there are still many enterprises, especially smaller companies that do not use discrete-event simulation because of the high costs associated with buying and maintaining commercial simulation tools. The question is whether or not free discrete-event simulation software is an alternative to commercial tools for solving typical planning tasks in production and logistics. The paper analyzes the modeling process with the three free and open-source discrete-event simulation tools Salabim, JaamSim and CloudSim and compares them with the two standard commercial simulation packages Arena and Plant Simulation. JaamSim provides everything which is necessary to model typical planning tasks in production and logistics and proves as a real alternative to commercial discrete-event simulation tools.

The integration of material flow simulation and digital planning solutions raises new challenges for the methodology and technical implementation of simulation models. The paper describes the use of a material flow simulation, which is controlling an AGV and interacting with digital twins of manufacturing cells. The digital twins determine the exact machining times. In addition, a mixed-reality application visualizes the entire system, while the communication via MQTT ensures a quasi-synchronous behavior of the different models and the AGV. The paper provides a literature review on similar solutions, describes the concept of our approach and specifies the technical implementation.

The article investigates the application of NeuroEvolution of Augmenting Topologies (NEAT) to generate and parameterize artificial neural networks (ANN) on determining allocation and sequencing decisions in a two-stage hybrid flow shop scheduling environment with family setup times. NEAT is a machine-learning and neural architecture search algorithm, which generates both, the structure and the hyper-parameters of an ANN. Our experiments show that NEAT can compete with state-of-the-art approaches in terms of solution quality and outperforms them regarding computational efficiency. The main contributions of this article are: (i) A comparison of five different strategies, evaluated with 14 different experiments, on how ANNs can be applied for solving allocation and sequencing problems in a hybrid flow shop environment, (ii) a comparison of the best identified NEAT strategy with traditional heuristic and metaheuristic approaches concerning solution quality and computational efficiency.

This paper aim is to provide a decision support model to assist the invoice delivery for about 200 customers in a law firm in Bogotá, Colombia. Each customer has a service time and establishes the conditions for the delivery, i.e., day, place, and time window. Invoices are delivered by motorcycle. We collect and analyze real travel and service times for each day of the week. To determine an optimal plan for invoice delivery, a mathematical model was developed for dealing with customer requests. One week (5 days) was chosen as a planning period for invoice delivery. Comparing the current route planning approach against the proposed method, we obtain savings in the total operating time by 13.3 %. Results indicate that the implementation of the proposed optimization model in weekly planning decisions could decrease operating times and increase service quality significantly.

The following paper presents the application of Deep Q-Networks (DQN) for solving a flexible job shop problem with integrated process planning. DQN is a deep reinforcement learning algorithm, which aims to train an agent to perform a specific task. In particular, we train two DQN agents in connection with a discrete-event simulation model of the problem, where one agent is responsible for the selection of operation sequences, while the other allocates jobs to machines. We compare the performance of DQN with the GRASP metaheuristic. After less than one hour of training, DQN generates schedules providing a lower makespan and total tardiness as the GRASP algorithm. Our first investigations reveal that DQN seems to generalize the training data to other problem cases. Once trained, the prediction and evaluation of new production schedules requires less than 0.2 seconds.

The paper shows how assigning different predefined dispatching rules for a predefined number of times at any points in time using a genetic algorithm can solve a hybrid flow shop scheduling problem with sequence-dependent setup times, using the example of a company producing printed circuit boards. The paper describes the implementation of the genetic algorithm, analyses the results for four different data sets and compares them with the results of applying standard dispatching rules. Using a genetic algorithm to assign dispatching rules achieves better solutions than simple dispatching rules often used in industrial practice. The genetic algorithm already achieves good solutions after a few seconds up to a few minutes, which means that this approach can be used for real-time decision making in a cyber-physical system in the age of Industry 4.0.

The paper proposes a simulation-based hyperheuristics approach to generate schedules for a two-stage hybrid flow shop scheduling problem with sequence-dependent setup times. The scheduling problem is derived from a company that is assembling printed circuit boards. A genetic algorithm determines sequences of standard dispatching rules that are evaluated by a discrete-event simulation model minimizing a multi-criteria objective composed of makespan and total tardiness. To reduce the computation time of the algorithm a dispatching rule-based chromosome representation is used containing a sequence of dispatching rules and time intervals in which the rules are applied. Different experiment configurations and their impact on solution quality and computation time are analyzed. The optimization model generates efficient schedules for multiple real-world data sets.

Automated guided vehicles (AGVs) form a large and important part of logistic systems to improve productivity and reduce costs. When multiple AGVs are running in limited and uncertain environments, lots of issues can occur, such as collisions and deadlocks, which need to be addressed. This paper presents a flexible simulation model for a warehouse with various AGVs. We implemented all three typical strategies to handle deadlocks: prevention, avoidance and detection and resolution. The results show that there is no dominant strategy and that the results strongly depend on the individual case and the input parameters.

When planning logistics systems with multiple transport objects or systems, modeling requires the implementation of complex control logic to avoid collisions and deadlocks. This paper illustrates a procedure for the development of such control logic on the example of rail-based storage and retrieval units in combinations with lifts in the picking area of an industrial laundry. Typical collision situations and a possible solution approach for avoiding these are described. In addition, a discrete event simulation demonstrates which situations occur most frequently depending on warehouse dimensioning.

Ongoing market concentration and a shortage of low-cost labor is compelling the German industrial laundry sector to adopt innovated automated solutions. RFID tags in textiles facilitate the use of automated systems. This paper introduces a new approach to picking and storing clean laundry. We simulate the system at an early stage to detect potential flaws during the development phase. The consortium plans to have a working prototype by the end of 2019.

Simulation models are still often only part for decision support in the planning area. For short-term decisions at the operational level, there have been good fundamentals since the 1990s, but still relatively few implementations, especially in the logistics sector. Our approach is to use real-time data to provide short-term forecasts, by using a simulation model that provides required information. Due to current hardware and a well-chosen degree of abstraction of the model, real-time decision support (“real-time” means in this context: fast enough to support the decision) is possible. This paper presents a concept of a procedure model for the realization of such operational simulation-based decision support, applied to the picking area of an industrial laundry. The operational use of the simulation model is part of the project “Laundry Order Consolidation System (LOCSys)”, which aims to improve the picking & storing processes in the clean area of an industrial laundry through automation.

Industrial laundries in Germany face a high pressure to increase their efficiency due to an ongoing market concentration. While many processes in an industrial laundry are already highly automated and optimized, order picking is still done manually. As RFID proliferates in industrial laundries, goods become traceable and identifiable. This enables the effective use of automated picking systems. Automation systems in the laundry industry face the challenge of pliable (form unstable) goods. This paper shows how the planning and dimensioning of an automated Laundry Order Consolidation System (LOCSys) can be supported by simulation modelling, already beginning in the early conception stage of the system.