Recent meta-learning approaches are oriented towards algorithm selection, optimization or recommendation of existing algorithms. In this paper we show how inductive algorithms constructed from building blocks on small data subsample can be scaled up to model large data sets. We demonstrate how one particular template (simple ensemble of fast sigmoidal regression models) outperforms state-of-the-art approaches on the Airline data set.

In this paper we present a novel algorithm called GPAT (Genetic Programming of Augmenting Topologies) which evolves Genetic Programming (GP) trees in a similar way as a well-established neuro-evolutionary algorithm NEAT (NeuroEvolution of Augmenting Topologies) does. The evolution starts from a minimal form and gradually adds structure as needed. A niching evolutionary algorithm is used to protect individuals of a variable complexity in a single population. Although GPAT is a general approach we employ it mainly to evolve artificial neural networks by means of Hypercube-based indirect encoding which is an approach allowing for evolution of large-scale neural networks having theoretically unlimited size. We perform also experiments for directly encoded problems. The results show that GPAT outperforms both GP and NEAT taking the best of both.

Agent-based models are helpful to investigate complex dynamics in coupled human–natural systems. However, model assessment, model comparison and replication are hampered to a large extent by a lack of transparency and comprehensibility in model descriptions. In this article we address the question of whether an ideal standard for describing models exists. We first suggest a classification for structuring types of model descriptions. Secondly, we differentiate purposes for which model descriptions are important. Thirdly, we review the types of model descriptions and evaluate each on their utility for the purposes. Our evaluation finds that the choice of the appropriate model description type is purpose-dependent and that no single description type alone can fulfil all requirements simultaneously. However, we suggest a minimum standard of model description for good modelling practice, namely the provision of source code and an accessible natural language description, and argue for the development of a common standard.

Recently published studies showed that age assessment methods are population specific. Authors analyse the senescence changes in pubic symphysis and sacro-pelvic surface of a pelvic bone using data mining methods. The multi-ethnic data set consists of 956 adult individuals ranging from 19 to 100 years of age derived from 9 different populations with known age and sex. The results show that accurate and reliable age assessment is possible to three age classes (less than 30, 30-60, 60 and more). The study confirms that population specificity of the methods exists and the variable "sex" is not important in age classification.

In this paper we propose a new algorithm called HyperGPEFS (HyperGP with Explicit Fitness Sharing). It is based on a HyperNEAT, which is a well-established evolutionary method employing indirect encoding of artificial neural networks. Indirect encoding in HyperNEAT is realized via special function called Compositional and Pattern Producing Network (CPPN), able to describe a neural network of arbitrary size. CPPNs are represented by network structures, which are evolved by means of a slightly modified version of another, well-known algorithm NEAT (NeuroEvolution of Augmenting Topologies). HyperGP is a variant of HyperNEAT, where the CPPNs are optimized by Genetic Programming (GP). Published results reported promising improvement in the speed of convergence. Our approach further extends HyperGP by using fitness sharing to promote a diversity of a population. Here, we thoroughly compare all three algorithms on six different tasks. Fitness sharing demands a definition of a tree distance measure. Among other five, we propose a generalized distance measure which, in conjunction with HyperGPEFS, significantly outperforms HyperNEAT and HyperGP on all, but one testing problems. Although this paper focuses on indirect encoding, the proposed distance measures are generally applicable.

Optimization of neural network topology, weights and neuron transfer functions for given data set and problem is not an easy task. In this article, we focus primarily on building optimal feed-forward neural network classifier for i.i.d. data sets. We apply meta-learning principles to the neural network structure and function optimization. We show that diversity promotion, ensembling, self-organization and induction are beneficial for the problem. We combine several different neuron types trained by various optimization algorithms to build a supervised feed-forward neural network called Group of Adaptive Models Evolution (GAME). The approach was tested on a large number of benchmark data sets. The experiments show that the combination of different optimization algorithms in the network is the best choice when the performance is averaged over several real-world problems.

Modern buildings are becoming more intelligent every day which helps preventing the waste of energy in cases when use of energy is not necessary. The problem of energy efficiency is in the control of power consumption. Controller has to find the balance between comfort of inhabitants and energy costs. Usually this dilemma is solved by hard wired rules that control energy consumption. These rules have to be analyzed and the system has to be set up, which is expensive. Even more expensive is the change of these rules requiring all these procedures to be processed again. All those problems can be solved by an application that uses network of sensors and actuators to analyze the state of building and gathers all the information from history to analyze inhabitants' behavioral patterns. According to behavioral patterns, the system can predict future steps of inhabitants to increase their comfort as well as find the balance between comfort and consumption. Our application Teiresias [13] employs software developed at our faculty called GAME [12] based on data mining and neural networks to find those patterns. The KNX [10] was chosen as a peripherals network for data gathering. The integration of KNX and GAME was a sufficient basis on which we tested scenarios where we have proven the concept of prediction based on gathered history data. We have theoretically proven the ability of Teiresias to predict inhabitants' needs and easily adapt to behavioral pattern changes. Examples of scenarios include control of heating, AC, ventilation, lights, sun blinds, watering, etc.

In this paper we describe simulation of autonomous robots controlled by recurrent neural networks, which are evolved through indirect encoding using HyperNEAT algorithm. The robots utilize 180 degree wide sensor array. Thanks to the scalability of the neural network generated by HyperNEAT, the sensor array can have various resolution. This would allow to use camera as an input for neural network controller used in real robot. The robots were simulated using software simulation environment. In the experiments the robots were trained to drive with imaximum average speed. Such fitness forces them to learn how to drive on roads and avoid collisions. Evolved neural networks show excellent scalability. Scaling of the sensory input breaks performance of the robots, which should be gained back with re-training of the robot with a different sensory input resolution.

In this paper we present neuro-evolution of neural network controllers for mobile agents in a simulated environment. The controller is obtained through evolution of hypercube encoded weights of recurrent neural networks (HyperNEAT). The simulated agent’s goal is to find a target in a shortest time interval. The generated neural network processes three different inputs – surface quality, obstacles and distance to the target. A behavior emerged in agents features ability of driving on roads, obstacle avoidance and provides an efficient way of the target search.

In this paper we present application of genetic programming (GP) [1] to evolution of indirect encoding of neural network weights. We compare usage of original HyperNEAT algorithm with our implementation; in which we replaced the underlying NEAT with genetic programming. The algorithm was named HyperGP. The evolved neural networks were used as controllers of autonomous mobile agents (robots) in simulation. The agents were trained to drive with maximum average speed. This forces them to learn how to drive on roads and avoid collisions. The genetic programming lacking the NEAT complexification property shows better exploration ability and tends to generate more complex solutions in fewer generations. On the other hand, the basic genetic programming generates quite complex functions for weights generation. Both approaches generate neural controllers with similar abilities.

Most of Feature Ranking and Feature Selection approaches ca n be used for categorial data only. Some of them rely on statistical measures of the data, some are tailo red to a specific data mining algorithm (wrapper approach). In this paper we present new methods for feature ranking and s election obtained as a combination of the above mentioned approaches. The data mining algorithm (GAME) is designed fo r numerical data, but it can be applied to categorial data as well. It incorporates feature selection mechanisms and n ew methods, proposed in this paper, derive feature ranking from final data mining model. The rank of each feature selecte d by model is computed by processing correlations of outputs between neighboring model’s neurons in different w ays. We used four different methods based on fuzzy logic, certainty factors and simple calculus. The performance of t hese four feature ranking methods was tested on artificial data sets, on well known Ionosphere data set and on well known Housing data set with continuous variables. The results indicated that the method based on simple calculus approach was significantly worse than other three methods. These methods produce ranking consistent with recently publishe d studies.

An accurate model of ultrasound estimation of fetal weight (EFW) can help in decision if the cesarean childbirth is necessary. We collected models from various sources and compared their accuracy. These models were mostly obtained by standard techniques such as linear and nonlinear regression. The aim of the comparison was to recommend a model best fitting to data measured for Czech population. Alternatively, we generated several linear and non-linear models by using our method GAME from the computational intelligence domain. GAME models can be serialized into simple equations that are understandable by domain experts. In this contribution, we show that automatically generated GAME models are at least as accurate (in terms of root mean squared error and standard deviations of predictions) as the best model computed by means of (time and expert skills demanding) standard techniques.

Long term Holter monitoring is widely applied to patients with heart diseases. Many of those diseases are not constantly present in the ECG signal but occurs from time to time. To detect these infrequent problems the Holter long time ECG recording is recorded and analysed. There are many methods for automatic detection of irregularities in the ECG signal. In this paper we will comapare the Support Vector Machine (SVM), J48 decision tree (J48), RBF artificial neural network (RBF), Simple logistic function and our novel GAME neural network for detection of the Premature Ventricular Contractions. We will compare and discuss classification performance of mentioned methods. There are also very many features which describes the ECG signal therefore we will try to identify features important for correct classification and examine how the accuracy is affected with only selected features in training set.

Nowadays a Feature Ranking (FR) is commonly used method for obtaining information about a large data sets with various dimensionality. This knowledge can be used in a next step of data processing. Accuracy and a speed of experiments can be improved by this. Our approach is based on Artificial Neural Networks (ANN) instead of classical statistical methods. We obtain the knowledge as a by-product of Niching Genetic Algorithm (NGA) used for creation of a feedforward hybrid neural network called GAME. In this paper we present a behaviour of FeRaNGA (Feature Ranking method using Niching Genetic Algorithm(NGA)) during a learning process, especially in every layer of generated GAME network. We want to answer how important is NGA configuration and processing procedure for FR results because behaviour of GA is nondeterministic and thereby were results of FeRaNGA also indefinitive. This method ranks features depending on a percentage of processing elements that survived a selection process. Processing elements transforms parent input features to an output. The selection process is realized by means of NGA where units connected to the least significant features starve and fade from population. To obtain the best results and to find optimal configuration is behaviour of the FeRaNGA algortithm tested using various parameters of NGA and number of ensemble GAME models on well known artificial data sets.

Artificial Neural Network is a, well-known tool used for data modeling of systems. This paper focuses on so called TWEANNs (Toppology and Weight Evolving Artificial Neural Networks). TWEANNs are Evolutionary Algorithms (EAs) which evolve both topology and parameters (weights) of neural networks. Here, we concentrate on a, use of an indirect. developmental encoding which is an approach inspired by multi-cellular orgnisms development, from a single cell (zygote) known from Nature. We examine, multiple modifications of a known tree-based indirect developmental encoding: the Cellular Encoding. Grammatical Evolution (GE) is employed instead of Genetic Programming (GP) to optimize program trees. GE is advantageous mainly in the way it can handle constraints (as it evolves program trees which conform to a grammar prespecified using a BNF notation). Moreover, we employ GE's inner mechanisms to efficiently encode Neural Network parameters (weights and biases). The Cellular Encoding is a, neuron-centric approach, therefore an increased attention should be paid to a way in which a proper synaptic link is selected (prior to modification of its parameters). In this work, we compare three different link select ion schemes. The results of our investigations show that Our modifications of Cellular Encoding improve the ability to evolve real-valued Artificial Neural Networks.

In this paper we present a new self-organizing neural network called Temporal Hebbian Self-organizing Map (THSOM) suitable for modelling of temporal sequences. The network is based on Kohonen's Self-organizing Map, which is extended with a layer of full recurrent connections among the neurons. The layer of recurrent connections is trained with Hebb's rule. The recurrent layer represents temporal order of the input vectors. The THSOM brings a straightforward way of embedding context information in recurrent SOM using neurons with Euclidean metric and scalar product. The recurrent layer can be easily converted into a stochastic automaton (Markov Chain) generating sequences used for previous THSOM training. Finally, two real world examples of THSOM usage are presented. THSOM was applied to extraction of road network from GPS data and to construction of spatio-temporal models of spike train sequences measured in human brain in vivo.

Most common feature ranking methods are based on the statistical approach. This paper compare several statistical methods with new method for feature ranking derived from data mining process. This method ranks features depending on percentage of child units that survived the selection process. A child unit is a processing element transforming the parent input features to the output. After training, units are interconnected in the feedforward hybrid neural network called GAME. The selection process is realized by means of niching genetic algorithm, where units connected to least significant features starve and fade from population. Parameters of new FR algorithm are investigated and comparison among different methods is presented on well known real world and artificial data sets.

The amount of data produced by medicine diagnosis and other means constantly increases -- in both number of measurements and in number of dimensions. For many modeling or data mining methods this increase causes problems. First main problem is well known curse of dimensionality. The second is the amount of training data items which lengthens the training process. Both these problems reduces usability of modeling methods.The aim of this article is to study several data reduction techniques and test their influence on one particular inductive modeling method -- GAME -- developed in our department. Application of each method affecting the performance (accuracy) and learning time of the GAME modeling method has been studied.To obtain representative results several datasets has been tested -- for example well known Iris dataset or real-world application for medical data (e.g. EEG classification).