In this paper, we introduce an alternative approach to enhancing Multi-Agent Reinforcement Learning (MARL) through the integration of domain knowledge and attention-based policy mechanisms. Our methodology focuses on the incorporation of domain-specific expertise into the learning process, which simplifies the development of collaborative behaviors. This approach aims to reduce the complexity and learning overhead typically associated with MARL by enabling agents to concentrate on essential aspects of complex tasks, thus optimizing the learning curve. The utilization of attention mechanisms plays a key role in our model. It allows for the effective processing of dynamic context data and nuanced agent interactions, leading to more refined decision-making. Applied in standard MARL scenarios, such as the Stanford Intelligent Systems Laboratory (SISL) Pursuit and Multi-Particle Environments (MPE) Simple Spread, our method has been shown to improve both learning efficiency and the effectiveness of collaborative behaviors. The results indicate that our attention-based approach can be a viable approach for improving the efficiency of MARL training process, integrating domain-specific knowledge at the action level.

We present a self-adaptive genetic algorithm for the problem of predicting if a Medicare standardized payment to a physical therapist will be above or below the national median. The percentage of Americans 65 and over is expected to increase in the coming years, increasing the need for physical therapy services. As a result, accurate prediction of expected Medicare payments based on local factors will be of increasing importance. A self-adaptive genetic algorithm is an evolutionary algorithm in which some or all of the algorithm's parameters are evolved over the course of its execution. Self-adaptation is a useful tool both for improving the performance of evolutionary algorithms, as well as improving usability through lessening the amount of parameter tuning required of the algorithm's user. While other self-adaptive approaches tend to focus on self-adaptation of only a few parameters, our approach self-adapts all of the parameters related to crossover and mutation. We compare the performance of our self-adaptive genetic algorithm with that of logistic regression and a canonical genetic algorithm on the problem of predicting Medicare payments. Logistic regression is a commonly used benchmark for this type of problem and a canonical genetic algorithm is included to allow us to see if any performance costs arise from the self-adaptive mechanisms. Results show that our self-adaptive genetic algorithm is effective at the classification of Medicare standardized payments to physical therapists, achieving accuracies of over 93%. Performance remains strong with training sets as small as 5% of the full data set. The problem representation used by our method allows for the identification of the relevant features for classification which means that our approach is capable of simultaneously performing classification and feature selection.

Internet of Things (IoT) devices are common in today’s computer networks. These devices can be computationally powerful, yet prone to cybersecurity exploitation. To remedy these growing security weaknesses, this work proposes a new artificial intelligence method that makes these IoT networks safer through the use of autonomous, swarm-based cybersecurity penetration testing. In this work, the introduced Particle Swarm Optimization (PSO) penetration testing technique is compared against traditional linear and queue-based approaches to find vulnerabilities in smart homes and IoT networks. To evaluate the effectiveness of the PSO approach, a network simulator is used to simulate smart home networks of two scales: a small, home network and a large, commercial-sized network. These experiments demonstrate that the swarm-based algorithms detect vulnerabilities significantly faster than the linear algorithms. The presented findings support the case that autonomous and swarm-based penetration testing in a network could be used to render more secure IoT networks in the future. This approach can affect private households with smart home networks, settings within the Industrial Internet of Things (IIoT), and military environments.

This paper describes our investigation to determine whether undesirable health conditions of an automobile driver can be identified in real time solely by monitoring and assessing his/her driving behavior. The concept has great potential to reduce the accident rate on roadways, especially for young inexperienced drivers who may be suffering from chronic health conditions that when uncontrolled, can result in dangerous driving actions. Our approach involves building models of “normal” and “abnormal” driving by an individual through machine learning from observation (MLfO, or simply LfO). Conceptually, discrepancies between actual driving actions taken by a driver in real time and the actions prescribed by a model of her/his normal driving, and/or similarities to a model of his/her abnormal driving, could indicate a dangerous medical condition. If appropriate, the system could alert the driver and/or the appropriate authorities (e.g., EMTs, police, or parents if a minor) of the potential for danger. More specifically, our research created models of human driving through the use of an LfO system developed previously in our laboratory called Force-feedback Approach to Learning from Coaching and Observation with Natural and Experiential Training (Falconet). Time-stamped traces of actions taken by 12 human test subjects in a driving simulator were collected and used to create the models of human driving behavior through Falconet. Then the overall actions prescribed by the models (called the agents) were compared to the original traces to ascertain whether similarities and/or differences between the human test subject behaviors and the agent behaviors could be indicative of the target conditions. In our use case presented here, the target condition was Attention Deficit/Hyperactivity Disorder (ADHD), a condition that afflicts many driving age teenagers and which can be detrimental to safe driving when not under control through medication. The work described in this paper is exploratory in nature, with the objective of showing scientific feasibility. The results of extensive testing indicate that the agents created with the Falconet system produced promising results, being able to correctly characterize traces in up to nearly 82% of the test cases presented. Nevertheless, as is typical in such exploratory works, we found that much further work remains to be done before this concept becomes ready for commercial application. In this paper we describe the approach taken, the agents created and the extensive quantitative experiments conducted, as well as any insights learned. Areas of further research are also identified and discussed.

Background: Lumbar microdiskectomy is amongst the most common neurosurgical techniques. In Saskatoon, minimally invasive microdiskectomy using the MetrX tubular retractor system has become a routinely performed procedure. While the outcomes of microdiskectomy are known to be similar to open technique, long term outcomes have not been reported. Methods: We performed a retrospective study of 160 minimally invasive microdiskectomies. We excluded subjects with cauda equina syndrome, redo surgery, fusions, and multi-level decompressions. We used one-way ANOVA to compare VAS, ODI, SF36, and EQ5D scores at pre-operative, 6-week postoperative, and long-term timepoints. Results: The mean pre-operative back pain VAS score was 6.23+/−2.63, 6-week post-operative follow-up VAS was 3.21+/−2.49, and long-term follow-up VAS was 2.56+/−2.45. The mean preoperative leg pain VAS score was 7.66+/−1.99, 6-week follow-up VAS was 3.56+/−2.79, final follow-up VAS was 2.20+/−2.57. The mean preoperative ODI score was 60.41+/−13.97; falling to 32.54+/−20.57 at 6-week follow up, and further to 24.50+/−20.97 at long term follow up. The mean baseline EQ5D quality of life score was 46.4+/−18.1, 68.9+/−20.2 at 6-week follow up and 69.3+/−20.3 at final review. Data reached statistical significance. Conclusions: We report good outcomes for minimally invasive microdiskectomy that are as durable as published results using open technique.

Path planning, the task of finding an obstacle-avoiding, shortest-length route from source to destination is an interesting theoretical problem with numerous applications. We present an improved genetic algorithm for path planning in a continuous, largely unconstrained real-world environment. We introduce a new domain-specific crossover operator based on path intersections. We also implement a new path correction operator that eliminates obstacle collisions from a path, leading to a dramatic search improvement despite the conceptual simplicity of the correction. Finally, in place of a standard binary measure of obstacle collisions, we present a new optimization objective measuring the degree to which a path intersects obstacles. Due to these improvements, individually and in combination, our algorithm is able to solve scenarios that are considerably more complex and exist in a more general environment than those that appear in the literature. We demonstrate the utility of our system through testing onboard an autonomous micro aerial vehicle. Further, our approach demonstrates the utility of domain-specific genetic operators for path planning. We hypothesize that such operators may be beneficial in other domains.

We investigate the evolution and survival of decision making strategies for the Deterrence Game in a population of interacting agents. The Deterrence Game is a sequential, two player game that is used to study how player preference orderings affect the outcome in situations of conflict. Although there are twenty-four possible player preference orderings, only five are given serious theoretical attention in the deterrence literature. We use analytical and empirical approaches to investigate what are the characteristics that make a preference ordering competitive or uncompetitive and whether the five traditionally studied preference orderings are the most competitive for their roles. Results find that four out of the five traditionally studied preference orderings are highly competitive. The fifth preference ordering is analytically moderately competitive but empirically unsuccessful. Analysis of all twenty-four preference orderings indicate that there are additional preference orderings that behave identically to the traditional five and are competitive.

We investigate the evolution and survival of decision making strategies for the Deterrence Game in a population of interacting agents. The Deterrence Game is a sequential, two player game that is used to study how player preference orderings affect the outcome in situations of conflict. Although there are twenty-four possible player preference orderings, only five are given serious theoretical attention in the deterrence literature. We use analytical and empirical approaches to investigate what are the characteristics that make a preference ordering competitive or uncompetitive and whether the five traditionally studied preference orderings are the most competitive for their roles. Results find that four out of the five traditionally studied preference orderings are highly competitive. The fifth preference ordering is analytically moderately competitive but empirically unsuccessful. Analysis of all twenty-four preference orderings indicates that there are additional preference orderings that behave identically to the traditional five and are competitive.

Viele Neurochirurgen setzen Antikonvulsiva routinemäßig zur Prophylaxe von epileptischen Anfällen bei Tumorresektionen ein, auch bei Patienten, die noch nie einen Anfall hatten. Die neue Studie zeigt, dass ohne Anfallsprophylaxe in dieser Patientengruppe bei nur 3% aller Patienten nach der Entfernung eines Hirntumors klinisch relevante Krampfanfälle auftraten. Angesichts der Nebenwirkungen des häufig verwendeten Phenytoins stellt sich die Frage, ob die Praxis der generellen Prophylaxe gerechtfertigt ist.

The goal of this work is to gain a better understanding of the role that inter-agent variation plays in self-organizing systems. We develop both continuous and discrete models of a multi-agent coordination procedure based on response thresholds and use these models to analyze the average behavior of a system as well as examine the dynamics of single instances of the problem. Results indicate that variation in the behaviors of agents can lead to increased stability and more effective self-organization in cooperative multi-agent systems.

One of the expected benefits of multi-agent systems is their capacity for redundancy which is expected to promote system robustness. We examine the impact agents' probabilistic action has on generating redundancy in multi-agent systems with no direct inter-agent communication. We present a mathematical model of such a system, analyze the factors that influence it, and empirically examine its effects on multi-agent systems under perturbations such as removal and insertion of agents. Results indicate that agents' probabilistic action can allow a team of collaborating agents to build a pool of agents with experience and this redundant pool can promote system robustness under perturbations.

Genetic algorithms (GAs) are probabilistic search techniques inspired by natural evolution. Selection schemes are used by GAs to choose individuals from a population to breed the next generation. Proportionate, ranking and tournament selection are standard selection schemes. They focus on choosing individuals with high fitness values. Fitness Uniform Selection Scheme (FUSS) is a recently proposed selection scheme that focuses on fitness diversity. FUSS have shown better performance than standard selection schemes for deceptive and NP-complete problems. In general, it is difficult to determine whether a real-life problem is deceptive or not. However, there is no information about the relative performance of FUSS on non-deceptive problems. In this paper, the standard selection schemes mentioned above were compared to FUSS on two non-deceptive problems. A GA using FUSS was able to find high-fitness solutions faster than expected. Consequently, FUSS could be a good first-choice selection scheme regardless of whether a problem at hand is deceptive or not.

This work investigates the behavior of a distributed team of agents on a dynamic distributed task allocation problem. Previous work finds that a distributed decision making process can effectively assign tasks appropriately to team members even when agents have only local information. We study this problem in a distributed environment in which agents can move, thus causing local neighborhoods to change over time. Results indicate that a higher level of adaptation is clearly required in the dynamic environment. Despite the increased difficulty, the distributed team is able achieve comparable behavior in both static and dynamic environments.

A recent theoretical investigation of modular representations shows that certain modularizations can introduce a distance bias into a landscape. This was a static analysis, and empirical investigations were used to connect formal results to performance. Here we replace this experimentation with an introductory runtime analysis of performance. We study a base-line, unbiased modularization that makes use of a complete module set (CMS), with special focus on strings that grow logarithmically with the problem size. We learn that even unbiased modularizations can have profound effects on problem performance. Our (1+1) CMS-EA optimizes a generalized OneMax problem in Ω( n 2 ) time, provably worse than a (1+1) EA. More generally, our (1+1) CMS-EA optimizes a particular class of concatenated functions in O (2 lm k n ) time, where l m is the length of module strings and k is the number of module positions, when the modularization is aligned with the problem separability. We compare our results to known results for traditional EAs, and develop new intuition about modular encapsulation. We observe that search in the CMS-EA is essentially conducted at two levels (intra- and extra-module) and use this observation to construct a module trap, requiring super-polynomial time for our CMS-EA and O ( n ln n ) for the analogous EA.

We are continuously challenged by ever increasing problem complexity and the need to develop algorithms that can solve complex problems and solve them within a reasonable amount of time. Modularity is thought to reduce problem complexity by decomposing large problems into smaller and less complex subproblems. In practice, introducing modularity into evolutionary algorithm representations appears to improve search performance; however, how and why modularity improves performance is not well understood. In this thesis, we seek to better understand the effects of modularity on search. In particular, what are the effects of module creation on the search space structure and how do these structural changes affect performance? We define a theoretical and empirical framework to study modularity in evolutionary algorithms. Using this framework, we provide evidence of the following. First, not all types of modularity have an effect on search. We can have highly modular spaces that in essence are equivalent to simpler non-modular spaces. This is the case, because these spaces achieve higher degree of modularity without changing the fundamental structure of the search space. Second, for the cases when modularity actually has an effect on the fundamental structure of the search space, if left without guidance, it would only crowd and complicate the space structure resulting in a harder space for most search algorithms. Finally, we have the case when modularity not only has an effect in the search space structure, but most importantly, module creation can be guided by problem domain knowledge. When this knowledge can be used to estimate the value of a module in terms of its contribution toward building the solution, then modularity is extremely effective. It is in this last case that creating high value modules or low value modules has a direct and decisive impact on performance. The results presented in this thesis help to better understand, in a principled way, the effects of modularity on search. Better understanding the effects of modularity on search is a step forward in the larger issue of evolutionary search applied to increasingly complex problems.

This paper focuses on the evolution of Fuzzy ARTMAP neural network classifiers, using genetic algorithms, with the objective of improving generalization performance (classification accuracy of the ART network on unseen test data) and alleviating the ART category proliferation problem (the problem of creating more than necessary ART network categories to solve a classification problem). We refer to the resulting architecture as GFAM. We demonstrate through extensive experimentation that GFAM exhibits good generalization and is of small size (creates few ART categories), while consuming reasonable computational effort. In a number of classification problems, GFAM produces the optimal classifier. Furthermore, we compare the performance of GFAM with other competitive ARTMAP classifiers that have appeared in the literature and addressed the category proliferation problem in ART. We illustrate that GFAM produces improved results over these architectures, as well as other competitive classifiers.

Modularity is thought to improve the evolvability of biological systems [18, 22]. Recent studies in the field of evolutionary computation show that the use of modularity improves performance and scalability of evolutionary algorithms for certain applications. [5, 12, 15, 16, 17]. The effects of introducing modularity to evolutionary search, however, are not well understood. This paper focuses on analyzing the effects of modularity on evolutionary computation. In particular, we analyze the effects of modular representations on the search space bias.

Fuzzy ARTMAP (FAM) is currently considered to be one of the premier neural network architectures in solving classification problems. One of the limitations of fuzzy ARTMAP that has been extensively reported in the literature is the category proliferation problem. That is fuzzy ARTMAP has the tendency of increasing its network size, as it is confronted with more and more data, especially if the data is of noisy and/or overlapping nature. To remedy this problem a number of researchers have designed modifications to the training phase of fuzzy ARTMAP that had the beneficial effect of reducing this phenomenon. In this paper we propose a new approach to handle the category proliferation problem in fuzzy ARTMAP by evolving trained FAM architectures. We refer to the resulting FAM architectures as GFAM We demonstrate through extensive experimentation that an evolved FAM (GFAM) exhibits good generalization, small size, and produces an optimal or a good sub-optimal network with a reasonable computational effort. Furthermore, comparisons of the GFAM with other approaches, proposed in the literature, that address the FAM category proliferation problem, illustrate that the GFAM has a number of advantages (i.e. produces smaller or equal size architectures, of better or as good generalization, with reduced computational complexity).

Many recent computer attacks have been launched in multiple stages to evade the detection of existing intrusion detection systems (IDS). Some stages of the attack may appear innocent if checked separately. Furthermore, the intervals between these separate attack stages can be on the order of hours, days, or even months. These characteristics of multi-stage attacks make the detection task challenging for most existing IDSs that are stateless in that they perform intrusion detection by independently checking individual packets, connections or sessions. In this paper, we propose a novel approach, active event correlation (AEC), which collects and correlates suspicious network events inside a network intrusion detection system (NIDS). AEC infers the possibility of attacks in the context of security policies and blocks attacks before they are completed. We have implemented AEC on top of the Bro NIDS (Paxson, 1999). Experiments indicate that AEC can effectively recognize and correlate individual stages of multi-stage attacks, stop incomplete attack stages, and give network administrators meaningful and concise alerts