An originally chaotic system can be controlled into various periodic dynamics. When it is implemented into a legged robot’s locomotion control as a central pattern generator (CPG), sophisticated gait patterns arise so that the robot can perform various walking behaviors. However, such a single chaotic CPG controller has difficulties dealing with leg malfunction. Specifically, in the scenarios presented here, its movement permanently deviates from the desired trajectory. To address this problem, we extend the single chaotic CPG to multiple CPGs with learning. The learning mechanism is based on a simulated annealing algorithm. In a normal situation, the CPGs synchronize and their dynamics are identical. With leg malfunction or disability, the CPGs lose synchronization leading to independent dynamics. In this case, the learning mechanism is applied to automatically adjust the remaining legs’ oscillation frequencies so that the robot adapts its locomotion to deal with the malfunction. As a consequence, the trajectory produced by the multiple chaotic CPGs resembles the original trajectory far better than the one produced by only a single CPG. The performance of the system is evaluated first in a physical simulation of a quadruped as well as a hexapod robot and finally in a real six-legged walking machine called AMOSII. The experimental results presented here reveal that using multiple CPGs with learning is an effective approach for adaptive locomotion generation where, for instance, different body parts have to perform independent movements for malfunction compensation.

Predictive feedback control is an easy-to-implement method to stabilize unknown unstable periodic orbits in chaotic dynamical systems. Predictive feedback control is severely limited because asymptotic convergence speed decreases with stronger instabilities which in turn are typical for larger target periods, rendering it harder to effectively stabilize periodic orbits of large period. Here, we study stalled chaos control, where the application of control is stalled to make use of the chaotic, uncontrolled dynamics, and introduce an adaptation paradigm to overcome this limitation and speed up convergence. This modified control scheme is not only capable of stabilizing more periodic orbits than the original predictive feedback control but also speeds up convergence for typical chaotic maps, as illustrated in both theory and application. The proposed adaptation scheme provides a way to tune parameters online, yielding a broadly applicable, fast chaos control that converges reliably, even for periodic orbits of large period.

Fruit flies (Drosophila melanogaster) rely on their olfactory system to process environmental information. This information has to be transmitted without system-relevant loss by the olfactory system to deeper brain areas for learning. Here we study the role of several parameters of the fly's olfactory system and the environment and how they influence olfactory information transmission. We have designed an abstract model of the antennal lobe, the mushroom body and the inhibitory circuitry. Mutual information between the olfactory environment, simulated in terms of different odor concentrations, and a sub-population of intrinsic mushroom body neurons (Kenyon cells) was calculated to quantify the efficiency of information transmission. With this method we study, on the one hand, the effect of different connectivity rates between olfactory projection neurons and firing thresholds of Kenyon cells. On the other hand, we analyze the influence of inhibition on mutual information between environment and mushroom body. Our simulations show an expected linear relation between the connectivity rate between the antennal lobe and the mushroom body and firing threshold of the Kenyon cells to obtain maximum mutual information for both low and high odor concentrations. However, contradicting all-day experiences, high odor concentrations cause a drastic, and unrealistic, decrease in mutual information for all connectivity rates compared to low concentration. But when inhibition on the mushroom body is included, mutual information remains at high levels independent of other system parameters. This finding points to a pivotal role of inhibition in fly information processing without which the system efficiency will be substantially reduced.

Since chaos control has found its way into many applications, the development of fast, easy-to-implement and universally applicable chaos control methods is of crucial importance. Predictive feedback control has been widely applied but suffers from a speed limit imposed by highly unstable periodic orbits. We show that this limit can be overcome by stalling the control, thereby taking advantage of the stable directions of the uncontrolled chaotic map. This analytical finding is confirmed by numerical simulations, giving a chaos-control method that is capable of successfully stabilizing periodic orbits of high period.

Memory storage in the brain relies on mechanisms acting on time scales from minutes, for long-term synaptic potentiation, to days, for memory consolidation. During such processes, neural circuits distinguish synapses relevant for forming a long-term storage, which are consolidated, from synapses of short-term storage, which fade. How time scale integration and synaptic differentiation is simultaneously achieved remains unclear. Here we show that synaptic scaling - a slow process usually associated with the maintenance of activity homeostasis - combined with synaptic plasticity may simultaneously achieve both, thereby providing a natural separation of short-from long-term storage. The interaction between plasticity and scaling provides also an explanation for an established paradox where memory consolidation critically depends on the exact order of learning and recall. These results indicate that scaling may be fundamental for stabilizing memories, providing a dynamic link between early and late memory formation processes.

Classical conditioning (conventionally modeled as correlation-based learning) and operant conditioning (conventionally modeled as reinforcement learning or reward-based learning) have been found in biological systems. Evidence shows that these two mechanisms strongly involve learning about associations. Based on these biological findings, we propose a new learning model to achieve successful control policies for artificial systems. This model combines correlation-based learning using input correlation learning (ICO learning) and reward-based learning using continuous actor–critic reinforcement learning (RL), thereby working as a dual learner system. The model performance is evaluated by simulations of a cart-pole system as a dynamic motion control problem and a mobile robot system as a goal-directed behavior control problem. Results show that the model can strongly improve pole balancing control policy, i.e., it allows the controller to learn stabilizing the pole in the largest domain of initial conditions compared to the results obtained when using a single learning mechanism. This model can also find a successful control policy for goal-directed behavior, i.e., the robot can effectively learn to approach a given goal compared to its individual components. Thus, the study pursued here sharpens our understanding of how two different learning mechanisms can be combined and complement each other for solving complex tasks.

Event Abstract Back to Event Information processing in the Drosophila Olfactory System: From Odors to Kenyon cells Faramarz Faghihi1*, Florentin Wörgötter1 and Christoph Kolodziejski1 1 Georg-August-University Goettingen, Drittes Physikalisches Institut, Germany Insect navigation in natural environments, for instance to seek food or to find a mate of the same species, relies on the efficiency of the insects’ olfactory system to detect, memorize, associate and retrieve olfactory information. The olfactory system of Drosophila (including Antennal Lobe and the Mushroom Body) consists only around 3000 neurons (Newquist, 2011) and is thus an ideal model to study the information processing of learning and memory (Masse et al. 2009). Although the system is very simple, experimentally assessing all parameters is still very difficult if not impossible. An important example for information processing is the threshold of coincidence detection in Mushroom Body neurons, the so called Kenyon cells. Here, coincidence detection means that a neuron can detect the occurrence of timely simultaneous but spatially separate input signals. In computational studies (Nowotny et al., 2003; Smith et al., 2008) the number of coincidentally active input neurons that suffice to trigger Kenyon cell firing (i.e. the threshold) is usually set arbitrarily and to our knowledge there exists only one study that addresses this question theoretically (Huerta and Garcia-Sanchez, 2003). As Kenyon cells are involved in associative odor learning (Galili et al., 2011), a detailed understanding of their function and parameters is advantageous for a better understanding of learning and memory in Drosophila. The quality of information processing in a computational model of the olfactory system can give us a lead on physiological and structural parameters of this system. For this purpose we use an information theoretical measure, the mutual information, which relates to the capacity of information transmission (Paninski, 2003). The greater the mutual information between, for instance, the presented odors and the Kenyon cells, the more information about the presented odor(s) is in the Kenyon cell firing and, thus, the more efficacious is the information processing. When we model the olfactory system with experimentally verified parameters and the environment in such a way that the system has to rely on the discrimination of distinct odors, we find, for instance, a clear optimum in mutual information for coincidence threshold values between 3 and 5. Additionally, non-synaptic modulatory input (Szyszka et al., 2008) modifies the optimal threshold values and can by this compensate dynamic environments or changing motivational states. This study shows the potential of using mutual information to extract basic structural and functional properties of such neuronal systems. Acknowledgements This research was supported by the BMBF-funded BCCN Goettingen with grant number 01GQ1005A, project B5. References Galili SD, Ludke A, Galizia CG, Szyszka P, Tanimoto H. Olfactory Trace Conditioning in Drosophila. Journal of Neuroscience 31(20): 7240-7248, 2011. Huerta R, Garcia-Sanchez M. Design parameters of the fan-out phase of sensory systems. Journal of Computational Neuroscience 15(1): 5-17, 2003. Masse NY, Turner GC, Jefferis GS. Olfactory information processing in Drosophila. Current Biology 19(16): R700-713, 2009. Newquist G. Brain organization and the roots of anticipation in Drosophila olfactory conditioning. Neuroscience Biobehavioral Review 35(5): 1166-1174, 2011. Nowotny T, Rabinovich MI, Huerta R, Abarbanel HD. Decoding temporal information through slow lateral excitation in the olfactory system of insects. Journal of Computational Neuroscience 15(2): 271-281, 2003. Paninski L. Estimation of Entropy and Mutual Information. Neural Computation 15: 1191–1253, 2003. Smith D, Wessnitzer J, Webb B. A model of associative learning in the mushroom body. Biological Cybernetics, 99(2): 89-103, 2008. Szyszka P, Galkin A, Menzel R. Associative and non-associative plasticity in Kenyon cells of the honeybee mushroom body. Frontiers in Systems Neuroscience, 2, 2008. Keywords: Drosophila Olfactory System, Information Processing, Kenyon cells, mutual information, Neuromodulatory Conference: Bernstein Conference 2012, Munich, Germany, 12 Sep - 14 Sep, 2012. Presentation Type: Poster Topic: Sensory processing and perception Citation: Faghihi F, Wörgötter F and Kolodziejski C (2012). Information processing in the Drosophila Olfactory System: From Odors to Kenyon cells. Front. Comput. Neurosci. Conference Abstract: Bernstein Conference 2012. doi: 10.3389/conf.fncom.2012.55.00189 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 11 May 2012; Published Online: 12 Sep 2012. * Correspondence: Mr. Faramarz Faghihi, Georg-August-University Goettingen, Drittes Physikalisches Institut, Goettingen, Niedersachsen, 37077, Germany, ffaghih2@gmu.edu Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Faramarz Faghihi Florentin Wörgötter Christoph Kolodziejski Google Faramarz Faghihi Florentin Wörgötter Christoph Kolodziejski Google Scholar Faramarz Faghihi Florentin Wörgötter Christoph Kolodziejski PubMed Faramarz Faghihi Florentin Wörgötter Christoph Kolodziejski Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Stabilizing unstable periodic orbits in a chaotic invariant set not only reveals information about its structure but also leads to various interesting applications. For the successful application of a chaos control scheme, convergence speed is of crucial importance. Here we present a predictive feedback chaos control method that adapts a control parameter online to yield optimal asymptotic convergence speed. We study the adaptive control map both analytically and numerically and prove that it converges at least linearly to a value determined by the spectral radius of the control map at the periodic orbit to be stabilized. The method is easy to implement algorithmically and may find applications for adaptive online control of biological and engineering systems.

In chaos control, an originally chaotic system is modified so that periodic dynamics arise. One application of this is to use the periodic dynamics of a single chaotic system as walking patterns in legged robots. In our previous work we applied such a controlled chaotic system as a central pattern generator (CPG) to generate different gait patterns of our hexapod robot AMOSII. However, if one or more legs break, its control fails. Specifically, in the scenario presented here, its movement permanently deviates from a desired trajectory. This is in contrast to the movement of real insects as they can compensate for body damages, for instance, by adjusting the remaining legs' frequency. To achieve this for our hexapod robot, we extend the system from one chaotic system serving as a single CPG to multiple chaotic systems, performing as multiple CPGs. Without damage, the chaotic systems synchronize and their dynamics is identical (similar to a single CPG). With damage, they can lose synchronization leading to independent dynamics. In both simulations and real experiments, we can tune the oscillation frequency of every CPG manually so that the controller can indeed compensate for leg damage. In comparison to the trajectory of the robot controlled by only a single CPG, the trajectory produced by multiple chaotic CPG controllers resembles the original trajectory by far better. Thus, multiple chaotic systems that synchronize for normal behavior but can stay desynchronized in other circumstances are an effective way to control complex behaviors where, for instance, different body parts have to do independent movements like after leg damage.

Conventional synaptic plasticity in combination with synaptic scaling is a biologically plausible plasticity rule that guides the development of synapses toward stability. Here we analyze the development of synaptic connections and the resulting activity patterns in different feed-forward and recurrent neural networks, with plasticity and scaling. We show under which constraints an external input given to a feed-forward network forms an input trace similar to a cell assembly (Hebb, 1949) by enhancing synaptic weights to larger stable values as compared to the rest of the network. For instance, a weak input creates a less strong representation in the network than a strong input which produces a trace along large parts of the network. These processes are strongly influenced by the underlying connectivity. For example, when embedding recurrent structures (excitatory rings, etc.) into a feed-forward network, the input trace is extended into more distant layers, while inhibition shortens it. These findings provide a better understanding of the dynamics of generic network structures where plasticity is combined with scaling. This makes it also possible to use this rule for constructing an artificial network with certain desired storage properties.

After only about 10 days would the storage capacity of our nervous system be reached if we stored every bit of input. The nervous system relies on at least two mechanisms that counteract this capacity limit: compression and forgetting. But the latter mechanism needs to know how long an entity should be stored: some memories are relevant only for the next few minutes, some are important even after the passage of several years. Psychology and physiology have found and described many different memory mechanisms, and these mechanisms indeed use different time scales. In this prospect we review these mechanisms with respect to their time scale and propose relations between mechanisms in learning and memory and their underlying physiological basis.

Neural systems regulate synaptic plasticity avoiding overly strong growth or shrinkage of the connections, thereby keeping the circuit architecture operational. Accordingly, several experimental studies have shown that synaptic weights increase only in direct relation to their current value, resulting in reduced growth for stronger synapses [1]. It is, however, difficult to extract from these studies unequivocal evidence about the underlying biophysical mechanisms that control weight growth. The theoretical neurosciences have addressed this problem by exploring mechanisms for synaptic weight change that contain limiting factors to regulate growth [2]. The effectiveness of these mechanisms is difficult to justify from a biophysical perspective, in particular those that require knowledge of global network status (e.g. knowledge of the ‘sum of all weights’) for normalization. Also spike-timing-dependent plasticity [3] cannot guaranty stability because various types of plasticity exist across different neurons and even at the same neuron, depending on the location of the synapses [1]. Therefore, it remains an open question how neural circuits simultaneously stabilize their many synapses and ensure diversity in the presence of a variety of distinct plasticity mechanisms. In 1998, a series of studies initiated by Turrigiano augmented this discussion by demonstrating that network activity is homeostatically regulated, suggesting that weights ω are regulated by an activity-dependent difference term [4,5]. Accordingly, synaptic scaling compares output activity v against a desired target activity vT of each individual neuron [5]. Most straightforwardly, such a local weight change is defined by dω/dt = γH(νT – ν) [6], where the long characteristic time scale (hours up to days) of synaptic scaling is determined by a small factor γ << 1. Synaptic scaling operates in parallel to conventional plasticity and acts simultaneously on different synapses. Here we suggest that synaptic scaling is combined with different types of plasticity mechanisms in the same circuit or even at the same neuron and regulates synaptic diversity across the circuit. We demonstrate that it robustly yields stable and diverse weight distributions which moreover are independent of the individual plasticity mechanism. As scaling co-acts with plasticity, such a combined mechanism is mathematically characterized by a weight change dω/dt = μG + γH. Here μ defines the rate of change of conventional synaptic plasticity, γ <<μ << 1, and G and H describe the specific types of plasticity and scaling, respectively [7]. For example, G is different for plain Hebbian plasticity than for STDP. As we show, combining any type of conventional plasticity G with nonlinear weight-dependent scaling H naturally yields global synaptic stabilization across the circuit regardless of the specific form of the plasticity G and also largely independent of the intrinsic neuron dynamics. Our study demonstrates that synapses are stabilized strictly in an input-determined way thereby capturing characteristic features of the inputs to the network. As an important result, we show that such systems are capable of representing a given input pattern via stably changed weights along several stages of signal propagation. This holds even in circuits containing a substantial number of random recurrent connections but no particular additional architecture.

It has been shown that plasticity is not a fixed property but, in fact, changes depending on the location of the synapse on the neuron and/or changes of biophysical parameters. Here, we investigate how plasticity is shaped by feedback inhibition in a cortical microcircuit. We use a differential Hebbian learning rule to model spike-timing-dependent plasticity and show analytically that the feedback inhibition shortens the time window for LTD during spike-timing-dependent plasticity but not for LTP. We then use a realistic GENESIS model to test two hypothesis about interneuron hypofunction and conclude that a reduction in GAD67 is the most likely candidate as the cause for hypofrontality as observed in Schizophrenia.

Synaptic scaling is a slow process that modifies synapses, keeping the firing rate of neural circuits in specific regimes. Together with other processes, such as conventional synaptic plasticity in the form of long term depression and potentiation, synaptic scaling changes the synaptic patterns in a network, ensuring diverse, functionally relevant, stable, and input-dependent connectivity. How synaptic patterns are generated and stabilized, however, is largely unknown. Here we formally describe and analyze synaptic scaling based on results from experimental studies and demonstrate that the combination of different conventional plasticity mechanisms and synaptic scaling provides a powerful general framework for regulating network connectivity. In addition, we design several simple models that reproduce experimentally observed synaptic distributions as well as the observed synaptic modifications during sustained activity changes. These models predict that the combination of plasticity with scaling generates globally stable, input-controlled synaptic patterns, also in recurrent networks. Thus, in combination with other forms of plasticity, synaptic scaling can robustly yield neuronal circuits with high synaptic diversity, which potentially enables robust dynamic storage of complex activation patterns. This mechanism is even more pronounced when considering networks with a realistic degree of inhibition. Synaptic scaling combined with plasticity could thus be the basis for learning structured behavior even in initially random networks.

In this article we exploit the discrete-time dynamics of a single neuron with self-connection to systematically design simple signal filters. Due to hysteresis effects and transient dynamics, this single neuron behaves as an adjustable low-pass filter for specific parameter configurations. Extending this neuro-module by two more recurrent neurons leads to versatile high- and band-pass filters. The approach presented here helps to understand how the dynamical properties of recurrent neural networks can be used for filter design. Furthermore, it gives guidance to a new way of implementing sensory preprocessing for acoustic signal recognition in autonomous robots.

Understanding closed loop behavioral systems is a non-trivial problem, especially when they change during learning. Descriptions of closed loop systems in terms of information theory date back to the 1950s, however, there have been only a few attempts which take into account learning, mostly measuring information of inputs. In this study we analyze a specific type of closed loop system by looking at the input as well as the output space. For this, we investigate simulated agents that perform differential Hebbian learning (STDP). In the first part we show that analytical solutions can be found for the temporal development of such systems for relatively simple cases. In the second part of this study we try to answer the following question: How can we predict which system from a given class would be the best for a particular scenario? This question is addressed using energy, input/output ratio and entropy measures and investigating their development during learning. This way we can show that within well-specified scenarios there are indeed agents which are optimal with respect to their structure and adaptive properties.

Network activity and network connectivity mutually influence each other. Especially for fast processes, like spike-timing-dependent plasticity (STDP), which depends on the interaction of few (two) signals, the question arises how these interactions are continuously altering the behavior and structure of the network. To address this question a time-continuous treatment of plasticity is required. However, this is - even in simple recurrent network structures - currently not possible. Thus, here we develop for a linear differential Hebbian learning system a method by which we can analytically investigate the dynamics and stability of the connections in recurrent networks. We use noisy periodic external input signals, which through the recurrent connections lead to complex actual ongoing inputs and observe that large stable ranges emerge in these networks without boundaries or weight-normalization. Somewhat counter-intuitively, we find that about 40% of these cases are obtained with a long-term potentiation-dominated STDP curve. Noise can reduce stability in some cases, but generally this does not occur. Instead stable domains are often enlarged. This study is a first step toward a better understanding of the ongoing interactions between activity and plasticity in recurrent networks using STDP. The results suggest that stability of (sub-)networks should generically be present also in larger structures.

In this theoretical contribution, we provide mathematical proof that two of the most important classes of network learningcorrelation-based differential Hebbian learning and reward-based temporal difference learningare asymptotically equivalent when timing the learning with a modulatory signal. This opens the opportunity to consistently reformulate most of the abstract reinforcement learning framework from a correlation-based perspective more closely related to the biophysics of neurons.

A confusingly wide variety of temporally asymmetric learning rules exists related to reinforcement learning and/or to spike-timing dependent plasticity, many of which look exceedingly similar, while displaying strongly different behavior. These rules often find their use in control tasks, for example in robotics and for this rigorous convergence and numerical stability is required. The goal of this article is to review these rules and compare them to provide a better overview over their different properties. Two main classes will be discussed: temporal difference (TD) rules and correlation based (differential hebbian) rules and some transition cases. In general we will focus on neuronal implementations with changeable synaptic weights and a time-continuous representation of activity. In a machine learning (non-neuronal) context, for TD-learning a solid mathematical theory has existed since several years. This can partly be transfered to a neuronal framework, too. On the other hand, only now a more complete theory has also emerged for differential Hebb rules. In general rules differ by their convergence conditions and their numerical stability, which can lead to very undesirable behavior, when wanting to apply them. For TD, convergence can be enforced with a certain output condition assuring that the δ-error drops on average to zero (output control). Correlation based rules, on the other hand, converge when one input drops to zero (input control). Temporally asymmetric learning rules treat situations where incoming stimuli follow each other in time. Thus, it is necessary to remember the first stimulus to be able to relate it to the later occurring second one. To this end different types of so-called eligibility traces are being used by these two different types of rules. This aspect leads again to different properties of TD and differential Hebbian learning as discussed here. Thus, this paper, while also presenting several novel mathematical results, is mainly meant to provide a road map through the different neuronally emulated temporal asymmetrical learning rules and their behavior to provide some guidance for possible applications.