Sequential temporal ordering and patterning are key features of natural signals used by the brain to decode stimuli and perceive them as sensory objects. To explore how cortical neuronal activity underpins sequence recognition, we developed a task in which mice distinguished between tactile ‘words’ constructed from distinct vibrations delivered to the whiskers, assembled in different orders. Animals licked to report the presence of the target sequence. Mice could respond to the earliest possible cues allowing discrimination, effectively solving the task as a ‘detection of change’ problem, but enhanced their performance when deliberating for longer. Optogenetic inactivation showed that both primary somatosensory ‘barrel’ cortex (S1bf) and secondary somatosensory cortex were necessary for sequence recognition. Two-photon imaging of calcium activity in S1bf layer 2/3 revealed that, in well-trained animals, neurons had heterogeneous selectivity to multiple task variables including not just sensory input but also the animal’s action decision and the trial outcome (presence or absence of a predicted reward). A large proportion of neurons were activated preceding goal-directed licking, thus reflecting the animal’s learnt response to the target sequence rather than the sequence itself; these neurons were found in S1bf as soon as mice learned to associate the rewarded sequence with licking. In contrast, learning evoked smaller changes in sensory responses: neurons responding to stimulus features were already found in naïve mice, and training did not generate neurons with enhanced temporal integration or categorical responses. Therefore, in S1bf sequence learning results in neurons whose activity reflects the learnt association between the target sequence and licking, rather than a refined representation of sensory features. ### Competing Interest Statement Michael Bale is now employed by Scientifica, manufacturer of the two-photon microscope used in the study.

Sequential temporal ordering and patterning are key features of natural signals, used by the brain to decode stimuli and perceive them as sensory objects. To explore how cortical neuronal activity underpins sequence discrimination, we developed a task in which mice distinguished between tactile "word" sequences constructed from distinct vibrations delivered to the whiskers, assembled in different orders. Animals licked to report the presence of the target sequence. Mice could respond to the earliest possible cues allowing discrimination, effectively solving the task as a "detection of change" problem, but enhanced their performance when responding later. Optogenetic inactivation showed that the somatosensory cortex was necessary for sequence discrimination. Two-photon imaging in layer 2/3 of the primary somatosensory "barrel" cortex (S1bf) revealed that, in well-trained animals, neurons had heterogeneous selectivity to multiple task variables including not just sensory input but also the animal's action decision and the trial outcome (presence or absence of the predicted reward). Many neurons were activated preceding goal-directed licking, thus reflecting the animal's learned action in response to the target sequence; these neurons were found as soon as mice learned to associate the rewarded sequence with licking. In contrast, learning evoked smaller changes in sensory response tuning: neurons responding to stimulus features were found in naive mice, and training did not generate neurons with enhanced temporal integration or categorical responses. Therefore, in S1bf, sequence learning results in neurons whose activity reflects the learned association between target sequence and licking rather than a refined representation of sensory features.

Mice discriminate tactile stimulus sequences distinguishable by the ordering of their elements. They detect changes between adjacent stimulus elements, focusing on the earliest cues allowing discrimination. Longer deliberation correlates with higher performance. Optogenetic suppression of barrel cortex (S1bf) or secondary somatosensory cortex (S2) reduces licking responses and abolishes discrimination, consistent with a loss of sensory input. Suppression of posterior parietal cortex (PPC) has no detectable effect. Suppression of whisker primary motor cortex (wM1) disinhibits lick responses. Two-photon imaging reveals that S1bf and PPC neurons show heterogeneous responses during sequence discrimination, reflecting sensory, decision/motor, and reward task variables. Our results are consistent with the decision to lick being achieved by a fast neuronal loop involving later processing stages. Upon learning a sequence, neurons in whisker somatosensory cortex link the target sequence with the associated (learnt) action.

To compare information and reach decisions effectively, our brain uses multiple heuristics, which can, however, induce biases in behavior. An elegant study by Akrami et al. (2018) finds evidence for one such heuristic in a sensory-based comparison task and identifies its location to the posterior parietal cortex.

The Barrels meeting annually brings together researchers focused on the rodent whisker to cortical barrel system prior to the Society for Neuroscience meeting. The 2017 meeting focused on the classification of cortical interneurons, the role interneurons have in shaping brain dynamics, and finally on the circuitry underlying oral sensations. The meeting highlighted the latest advancements in this rapidly advancing field.

Article Figures and data Abstract Introduction Results Discussion Materials and methods References Decision letter Author response Article and author information Metrics Abstract The world around us is replete with stimuli that unfold over time. When we hear an auditory stream like music or speech or scan a texture with our fingertip, physical features in the stimulus are concatenated in a particular order. This temporal patterning is critical to interpreting the stimulus. To explore the capacity of mice and humans to learn tactile sequences, we developed a task in which subjects had to recognise a continuous modulated noise sequence delivered to whiskers or fingertips, defined by its temporal patterning over hundreds of milliseconds. GO and NO-GO sequences differed only in that the order of their constituent noise modulation segments was temporally scrambled. Both mice and humans efficiently learned tactile sequences. Mouse sequence recognition depended on detecting transitions in noise amplitude; animals could base their decision on the earliest information available. Humans appeared to use additional cues, including the duration of noise modulation segments. https://doi.org/10.7554/eLife.27333.001 Introduction To make sense of the world around us, the brain must integrate sensory patterns and sequences over time and assign them meaning. Signals in our environment unfold over time and can only be interpreted by decoding their temporal patterning. The ability to do so underpins much of our sensory experience – for example, it is central to recognising a favourite melody or a passage of speech (Wilson et al., 2017). As first proposed over 60 years ago (Lashley, 1951), sequence processing provides a model for investigating how neuronal circuits give rise to object perception and recognition, a central goal of neuroscience (Griffiths and Warren, 2004; Dehaene et al., 2015). In tactile sensation, fast sensory events, such as fluctuations in the forces acting on a whisker follicle, are encoded faithfully and with high temporal precision (Johnson, 2001; Johansson and Birznieks, 2004; Jones et al., 2004; Chagas et al., 2013; Bale et al., 2015; Campagner et al., 2016; Bush et al., 2016). Exploring an object by scanning with fingertips or whiskers generates a series of tactile events concatenated over time (Phillips and Johnson, 1985; Phillips et al., 1990; Weber et al., 2013; Maravall and Diamond, 2014; Sofroniew and Svoboda, 2015; Saal et al., 2016). Recognising the object as a whole – its texture, shape or size – requires integrating over these events, with the relevant timescales varying from tens of milliseconds (ms) to seconds. Introspection suggests that meaningful auditory sequences, such as those in speech or music, can be learned quickly and robustly. We wondered whether similarly effective sequence learning and recognition occurs in tactile sensory systems. We further wished to explore the cues that could underlie capacities for tactile sequence recognition. To address these issues, we developed a new experimental design for testing sequence discrimination in mice and humans. Participants learn to distinguish a target stimulus sequence, constructed from an underlying noise waveform, from other stimuli that differ only in their temporal patterning over hundreds of milliseconds. Our results demonstrate efficient learning of tactile sequences both in mice and in humans. This behaviour provides an assay for exploring the neuronal circuit mechanisms that underpin recognition of temporally patterned stimuli. Results Achieving sequence recognition by mice We sought to train mice to recognise a target stimulation sequence delivered to their whiskers (Figure 1). Our aim was for mice to distinguish the target sequence based on the order in which its elements appeared. To this end, mice were trained to distinguish between initially meaningless GO and NO-GO sequences built from series of identical ‘syllables’, with the sequences differing only in that syllables were scrambled in time over hundreds of milliseconds (each individual syllable lasting 100 ms, for a total of 8 syllables; Figure 1D). The initial syllable was identical across GO and NO-GO sequences in order to avoid providing a stimulus onset cue (Figure 1D). Figure 1 Download asset Open asset Design of sequence recognition task for mice and humans. (A) Illustration of the treadmill-based behavioural and stimulus delivery setup for head-fixed mice. (B) Block diagram representing the structure of the GO/NO-GO paradigm. On GO trials, a mouse licking the water spout within the response period (hit) was rewarded with a water droplet. If the mouse licked on a NO-GO trial (false alarm), the next trial was delayed by 2–5 s. (C) Illustration of the stimulus delivery setup for human experiments. (D) Stimulus sequences for GO/NO-GO discrimination and intermediate ‘shaping’ sequences used for training. https://doi.org/10.7554/eLife.27333.002 Mice (n = 22) were trained to associate the GO stimulus with a water reward by making water available when the GO stimulus was delivered; on the first few days of training no other whisker stimuli were given, so that mice effectively learned to detect whisker stimulation. As soon as animals demonstrated detection (75% correct detection trials), we introduced an initial NO-GO sequence. To make this stage easier, this initial NO-GO sequence consisted of a square wave riding upon low amplitude noise, distinctly different from the GO sequence (Figure 1D, 'Square Wave'). Mice quickly learned to distinguish the square wave stimulus from the GO waveform (75% correct; within four sessions; Figure 2A). They were immediately moved to the next stage of training to avoid creating an artefactual generalized association between ‘noisy’ stimuli (as opposed to square waves) and water availability. In the following —more demanding— stage, the NO-GO sequence consisted of a scrambled GO sequence with half (4 of 8) syllables knocked out (Figure 1D, 'Half' NO-GO). Figure 2 Download asset Open asset Progression through sequence learning in mice. (A) Number of training sessions needed to learn different training stages (75% performance criterion). Each dot, one mouse. Dots are jittered for visualisation. (B) Performance metrics for an example training session (discrimination of the GO sequence from square wave). Correct trials are green dots, incorrect are red dots. Performance quantified as % correct over a 50-trial moving window (bottom). Whisker dependency on the task was verified by removing the actuator at trial 220 (fuchsia bar) and reinserting (blue bar) at trial 270. Apparent delay in performance is caused by 50-trial averaging. (C) Performance for an example session with stimulus rotation (GO sequence versus square wave discrimination). Main symbols as for B. Stimuli were delivered, as normal, first in the rostro-caudal axis (RC; blue bar) but following a brief period of stimulator removal (fuchsia bar), in the dorso-ventral (DV; cyan) axis for 50 trials. Stimulation then returned to RC for the remainder of the session. https://doi.org/10.7554/eLife.27333.003 Figure 2—source data 1 This zip archive includes all files used for generating the figure. Fig2.m is the Matlab script for plotting the data. m099_22-may-2015_alltrialdata_matrix.mat and 2987_22-feb-2016_alltrialdata_matrix.mat contain the trial by trial response outcomes used to construct the performance data in panels B and C. figure2a.mat, figure2b.mat and figure2c.mat contain all other Matlab variables needed for panels A, B and C respectively. https://doi.org/10.7554/eLife.27333.004 Download elife-27333-fig2-data1-v1.zip Mice accomplished each stage of training within a few days (Figure 2A), performing approximately 200–300 trials per daily session (mean 249 trials; SD 71 trials; total n = 456 sessions in 22 mice). During this process, recognition of the GO sequence was mediated by the animal’s whiskers: performance fell to chance level upon removing the whiskers from the moving stimulator (Figure 2B). Performance recognising the GO sequence was robust against variations in how the sequence was presented: daily changes in the tube’s positioning relative to the stimulated whisker did not noticeably affect performance. To test this invariance more specifically, in a subset of experiments mice were trained on a multi-whisker version of the task where whiskers (left untrimmed) were inserted into a wire mesh attached to the piezo actuator. Whiskers were first removed from the stimulator mesh; then, after a period of trials with stimulator movement but no whisker stimulation —during which performance dropped to chance level—, the actuator was rotated 90° and whiskers reinserted into the mesh. Reinsertion and stimulator rotation changed the identity and set-point of whiskers being stimulated as well as the direction of stimulation. Thus, the new configuration involved a different array of forces and moments acting upon a different set of whisker follicles. Yet performance quickly recovered to the level reached before whisker removal (Figure 2C; repeated for n = 4 mice; p=0.44; Wilcoxon signed rank test). Sequence recognition therefore transferred across different stimulation configurations, implying that the animal must be focusing on aspects of the pattern of fluctuations over time, rather than on movements of a specific whisker or in a specific direction. In the final stage of training, the NO-GO sequence comprised identical syllables to the full GO sequence but scrambled in time: that is, syllable ordering changed (Figure 1D). An example session from this final stage is shown in Figure 3. The animal’s performance on this session demonstrates consistent licking to the GO sequence and an ability to withdraw from licking impulsively to the NO-GO sequence. In this session, the rate of lick responses in a GO trial became greater than that in a NO-GO trial by 200 ms into the stimulus sequence (Figure 3D). This was 100 ms after the transition from the first to the second syllable, which –given our 100 ms resolution– was the earliest that an instantaneous ideal observer could have determined sequence identity. Figure 3 Download asset Open asset Performance of a trained mouse in a sequence recognition session. (A) Lick time raster plot for GO trials in an example session on the final stage of training (GO vs ‘full’ NO-GO). Licks are referenced to the start of the sequence, which lasts 800 ms (grey bar at top). The response period (fuchsia bar) opens after the end of stimulation. Water was only available in the response period. Hit and miss trials (rows) marked with green and red dots, respectively. (B) Lick time raster plot for NO-GO trials of same session as A. Green dots denote correct rejection trials, red dots false alarm trials. Other symbols as for A. (C) Smoothed average of lick rate signals per trial from A and B. Dark blue, GO trials. Teal, NO-GO. (D) Difference of lick rate plots. Continuous thick line is true difference between lick rates on GO and NO-GO trials from the best 50-trial moving window from C. Broken line is average of 50 repeats of differences between randomly drawn trials. Dashed threshold line is 95% confidence interval of lick rate values from the 50 repeats. Asterisk denotes the time when the true lick rate difference surpassed the 95% CI, termed discriminative lick latency. This is an upper bound on the animal’s time to discriminate sequence identity. https://doi.org/10.7554/eLife.27333.005 Figure 3—source data 1 This zip archive includes all files used for generating the figure. Fig3.m is the Matlab script for plotting the data. M058_12-Aug-2014_alltrialdata.mat contains trial by trial data including lick times, used to construct the lick rasters in panel A and B. Matlab script for making the lick rasters and histogram is in lick_rasters.m, and script for computing discrimination lick latency is in latency_analysis.m. https://doi.org/10.7554/eLife.27333.006 Download elife-27333-fig3-data1-v1.zip Mice that underwent training up to the final stage performed beyond 70% on at least one session (median 73.9%, SD 9.4%; n = 5 out of 6 mice). Animals could maintain their performance over several days, albeit with fluctuations (Figure 4A), again despite day-to-day variability in how the whisker was attached to the stimulator. On every stage of training, improvements in performance occurred mainly through learning to withhold impulsive false alarm responses (licks) (Figure 3C–D and 4B–C), in common with other discrimination tasks in mice (Guo et al., 2014; Berditchevskaia et al., 2016). Thus, high performance was associated with low false alarm rates (Figure 4C). Figure 4 Download asset Open asset Sequence recognition performance across mice. (A) Performance on full sequence recognition task across 8 successive behavioural sessions for one mouse. Performance is averaged over a 50-trial moving window. Colour corresponds to colour of same mouse in panels B-D. (B) Hit rate plotted against % correct for all sessions on full task. Each dot, one session on GO vs full NO-GO; each colour depicts a different mouse (n = 6). (C) False alarm rate plotted against % correct. Symbols as for B. (D) Discriminative lick latency plotted against % correct. https://doi.org/10.7554/eLife.27333.007 Figure 4—source data 1 This zip archive includes all files used for generating the figure. Fig4.m contains the Matlab script for constructing the figure. behaviour_fulltask_database.m contains an associated experiment database used for plotting the data. M058_fulltaskdata.mat is the session by session, trial by trial response outcome data used to calculate performance for the example mouse in panel A. fulltask_data_june.mat contains the performance for all mice used in panels B, C and D. fulltask_latency_data.mat contains the latency data for panel D. https://doi.org/10.7554/eLife.27333.008 Download elife-27333-fig4-data1-v1.zip In trained animals, therefore, the rate of licks on GO trials was higher than on NO-GO trials. The time elapsed from trial start to when the rates diverged provides a conservative upper bound on an animal’s decision time (Figure 3D); we refer to this as ‘discriminative lick latency’ (see Materials and methods). This latency varied from animal to animal and session to session (Figure 4D). For different animals, median discriminative lick latency ranged between 200–550 ms. That this value was at most 550 ms indicates that mice reached their decision on the identity of a sequence well before its end, as sequences lasted 800 ms. Performance on a session did not correlate with early or late detection of the sequence (Figure 4D; Spearman rho = - 0.033; p=0.88). These experiments show that mice learned to distinguish whisker-mediated stimuli that differed from others only in their temporal patterning over hundreds of milliseconds. They further indicate that mice based this behaviour on aspects of the pattern of fluctuations over time; in some cases, mice discriminated effectively almost as soon as the identity of the sequence was detectable. Rapid learning of noisy sequences by humans We tested whether humans could also learn to recognise the same temporally patterned tactile noise stimuli, delivering the patterns through an actuator applied to a fingertip. Participants were asked to indicate recognition of the target sequence by pressing a button. They first underwent a training session in which the GO target sequence —identical to that presented to mice— was interleaved with a series of non-target stimuli. Early in the training session, we presented non-target patterns set to be easily distinguishable from the GO pattern (e.g. sinusoidal waveforms). Later, these clearly distinct patterns were replaced by NO-GO sequences that differed from the GO target only in that their constituent syllables were scrambled, as in the final stage of mouse training. Human participants quickly improved their performance over the course of this session, typically converging to a steady level of performance despite the increase in difficulty during the session (Figure 5A). This indicated fast learning of the GO sequence. Performance as tested after the end of this learning phase was maintained in a later session separated by at least one week, suggesting remarkable robustness (Figure 5B; retest performance was actually higher, although the difference did not reach significance; p=0.0547; n = 11 participants; Wilcoxon signed rank test). Despite unavoidable differences in training procedure, mice and humans reached a similar median level of performance (mice: 74%; humans: 70%), and the range of performance across individuals also overlapped substantially between the two species (Figure 5B). Figure 5 Download asset Open asset Sequence recognition performance across humans. (A) Performance over the course of the first training session, measured as % correct averaged over 50-trial moving window. Grey lines, individual participants. Black line, average over participants. (B) Performance for best 50-trial window on initial test session after training and (on a subset of participants) upon retesting after a minimum of one week. Each grey dot, one human participant; black dots, participants tested on both sessions. Coloured dots, performance on individual sessions for mice, replotted from Figure 4 for comparison (each mouse, one colour; colours as for Figure 4). Dots are jittered along x axis for visualisation. https://doi.org/10.7554/eLife.27333.009 Figure 5—source data 1 This zip archive includes all files used for generating the figure. Fig5.m contains the Matlab script for making the figure. figure5a.mat and figure5b.mat contain the data used for constructing performance values in panels A and B. full_task_data_june.mat (same as for Figure 4—source data 1) contains performance values for all mice in the comparable experiment. colorblind_colormap.mat contains a colorblind appropriate colormap for displaying the mouse data. https://doi.org/10.7554/eLife.27333.010 Download elife-27333-fig5-data1-v1.zip Potential cues for sequence recognition Our experiments established that animals based their sequence discrimination behaviour on aspects of the pattern of fluctuations over time. This prompted us to wonder which cues were robust correlates of sequence identity, and which were used under our experimental conditions. In our task, possible cues ranged across timescales from global to local, as follows (Figure 6A). At the highest (most global) level the GO sequence could be recognised by extracting its overall ordering rule (i.e. distinguishing the [3 1 4 2 3 1 4 2] amplitude modulation envelope versus other scrambled orders). However, recognition could also stem from detection of specific syllables within the overall modulation envelope. For example, in the GO sequence [3 1 4 2 3 1 4 2] the second syllable was smaller in magnitude than the first, and this was in contrast to the NO-GO sequence [3 4 2 1 2 4 3 1], whose second syllable was greater than the first (Figure 1D). Thus, an animal could potentially base its recognition of the GO sequence on detecting a downwards modulation in noise amplitude after 100 ms. Finally, a strategy based on detecting even briefer, sub-syllabic events or ‘landmarks’ in the sequence, such as fluctuations in whisker velocity happening in a certain relative order (Figure 6A), could also be possible. We sought to identify cues and strategies accounting for the performance of mice and humans. Figure 6 Download asset Open asset Variants of task design to test for behavioural use of cues. (A) Cues within the GO sequence (black line) that could allow recognition of sequence identity. An example of a local cue (within green box) is the large isolated transient ‘landmark’ (green arrow) immediately followed by a low amplitude syllable. In contrast, global cues involve changes in integrated stimulus amplitude over time, as reflected in the amplitude modulation envelope (black dotted line). (B) Binary sequences distinguishable based on syllable ordering or the durations of small-amplitude epochs. Different colours indicate epochs of large and small noise amplitude. (C) Performance on task variants using binary sequences, compared to original GO vs full NO-GO design (leftmost data point, replotted from Figure 4 and 5 with same colour assignment). Each colour depicts a mouse. Black dots and error bars, grand average and SD for each task. Grey dots in labelled circle depict performance of individual human participants on [4 1 4 1 1 1 4 1] vs [4 1 1 1 4 1 4 1]. (D) Sequences used to test effect of fixed ‘landmarks’. Frozen GO used an identical waveform across syllables, trials and sessions. Unfrozen GO maintained the same sequence of noise amplitudes (indicated by colour coding) but varied the detailed waveform across syllables, trials and sessions. NO-GO scrambled the order of syllables, i.e. the sequence of noise amplitudes. (E) Hit rate of humans on frozen and unfrozen GO trials. Each dot, one participant and session. https://doi.org/10.7554/eLife.27333.011 Figure 6—source data 1 This zip archive includes all files used for generating the figure. Fig6.m is the Matlab script for constructing the figure. figure 6a.mat contains stimulus waveform data for making the schematic in panel A. noises_easy_apr16.mat contains the stimulus sequences for the task variants shown in panel B. figure6c_easy.mat contains mouse performance for the task variants in panel C. fulltask_data_june.mat (same as for Figure 4—source data 1) contains mouse performance data for the original task. data_fig6C_human.mat contains human performance data for panel C. figure6d.mat contains the frozen and unfrozen sequence waveforms in panel D. figure 6e.mat contains human hit rates on the task with frozen and unfrozen GO sequences, plotted in panel E. https://doi.org/10.7554/eLife.27333.012 Download elife-27333-fig6-data1-v1.zip First, our stimulus design ruled out cues based on overall temporal averaging or summing over stimulus parameters throughout the duration of the sequence, because GO and NO-GO sequences consisted of identical, but temporally scrambled elements. Second, mice reached their decision well before the end of sequence presentation (Figure 4D): as mentioned, discriminative lick latencies were sometimes consistent with fast detection of the first transitions or syllables that allowed sequence identification. Thus, just one or a few transitions in noise modulation, or the arrival of a small number of stimulus landmarks, could suffice for an animal to reach its decision. Binary sequence discrimination To further test the ability of mice to exploit specific cues for sequence recognition, we designed a version of the task in which the GO target sequence consisted of a simple succession of epochs of large and small noise amplitude: using the same notation as above, [4 1 4 1 1 1 4 1] (Figure 6B). A separate set of animals (n = 10) was trained to distinguish this new GO sequence from a NO-GO sequence that, as before, differed only in its temporal patterning: [4 1 1 1 4 1 4 1] (Figure 6B). These sequences were simpler than the original design in that they were binary: their constituent syllables were only ‘large’ (4) or ‘small’ (1). Note that the pattern design was such that the actual sequence of transitions in amplitude was identical between GO and NO-GO: transitions followed the order [−3 +3 –3 +3 –3]. Thus to distinguish these sequences, animals needed to focus on syllable ordering or on the different durations of small-amplitude epochs: for example, in the binary GO target sequence the first ‘small’ epoch lasted just 100 ms, but in the NO-GO it lasted for a total of 300 ms (because comprising three syllables). We found that mice did not perform well at distinguishing [4 1 4 1 1 1 4 1] from [4 1 1 1 4 1 4 1], despite having been trained exclusively on this variant of the task. Performance was above chance but never reached 75% (Figure 6C; n = 3), in contrast to the original variant. In particular, animals consistently displayed high false alarm rates, suggesting that they failed to detect what made the binary NO-GO stimulus different from the binary GO (data not shown). This suggested that in this paradigm mice could have had trouble either detecting the simpler, binary stimulus modulation epochs or recognising their differential duration. To distinguish between these possibilities, we tested performance on probe sessions with two variants of the binary NO-GO sequence. In the first, the binary GO target remained identical as [4 1 4 1 1 1 4 1], but the NO-GO sequence was [4 1 1 1 1 1 4 4]. This alternative NO-GO stimulus had the same number of large and small syllables as the one in the previous paragraph, but a different temporal arrangement, with all small-amplitude syllables appearing consecutively and forming a single very long central period (Figure 6B). The sequence therefore effectively had just two ‘large’ epochs, at the beginning and at the end, and a single very long ‘small’ epoch. There were now just two transitions in amplitude, in the order [−3 +3]. We therefore expected this NO-GO sequence to be easier to discriminate from the GO sequence despite having the same overall energy. In the second variant, the binary GO target also remained identical and the NO-GO stimulus was [4 1 1 1 1 1 1 1] (Figure 6B). The goal of this variant was to check whether mice could straightforwardly distinguish large or small syllables, as the NO-GO stimulus consisted mostly of small syllables and had lower overall energy than the (unchanged) GO stimulus. In both of these variants animals performed better than in the original binary design (Figure 6C; p=1.8×10−10; t-statistic for the regression on task type = 6.82; n = 10 mice and n = 161 sessions; generalised linear mixed-effects model). Note that performance in the simpler variants was at a level comparable to that of the original GO vs NO-GO paradigm (Figure 6C). Mice also successfully distinguished the binary GO sequence from noise stimuli with no modulation, i.e. with constant noise amplitude: [1 1 1 1 1 1 1 1] or [2 2 2 2 2 2 2 2] (percentage correct >75% for all animals for [1 1 1 1 1 1 1], 3 out of 4 for [2 2 2 2 2 2 2 2]; n = 4 mice; data not shown). These results indicate that mice in the binary sequence experiments could successfully detect ‘large’ epochs and recognise their number, and could use relative modulations (i.e. transitions) in noise amplitude as cues. However, mice trained in these experiments were not as successful at using the duration of modulation epochs as a cue. In contrast to mice, humans rapidly performed well on the [4 1 4 1 1 1 4 1] vs [4 1 1 1 4 1 4 1] version of the task (Figure 6C), even when they had had no prior exposure to the version in Figure 5 (5 out of 8 participants). They achieved high performance within one training session. They also subjectively reported being able to use the relative durations of ‘small’ and ‘large’ epochs as a cue. Thus, under these conditions humans appeared to readily access more cues for sequence discrimination than mice, including the duration and ordering of intervals in noise modulation. Fixed landmarks versus amplitude modulation Our findings suggested that humans could discriminate sequences based on multiple cues. We wondered whether, in addition to being sensitive to the size and timing of noise amplitude modulations, human participants might also rely on detecting learned sub-syllabic ‘landmarks’, that is, specific brief events or fluctuations within the stimulus waveform (Figure 6A). To address this, we assessed whether the presence of fixed waveforms influenced performance. Upon training participants on the initial version of the task (Figure 5), we tested performance on a variant of the design with two types of GO trials. The first type of trial used a target sequence constructed by applying amplitude modulation to a waveform that was identical (repeated) across syllables and trials (‘frozen’). This sequence was used throughout training and in the experiments of Figures 1–5. The second type of trial presented a sequence built by modulating a noise waveform that varied on every repeat (‘unfrozen’) (Figure 6D). For unfrozen sequences, each of the eight syllables was based on a different noise snippet and each trial was constructed from a fresh waveform. Thus, in this type of trial, the amplitude modulation envelope characteristic of the GO sequence remained identical across target trials, but not the precise stimulus values, so that sub-syllabic fluctuations were not conserved. Frozen and unfrozen GO trials were interleaved within a session. Note that unfrozen waveforms could vary in their empirical standard deviation, potentially leading to a confound caused by variability in perceived stimulus amplitude. To control for this, we included in our analysis only stimuli matched for empirical standard deviation. We compared hit rates for both types of GO trial (Figure 6E). Hit rates varied little across type of trial (frozen trials mean 0.76, SD 0.17; unfrozen tria

The world around us is replete with stimuli that unfold over time. When we hear an auditory stream like music or speech or scan a texture with our fingertip, physical features in the stimulus are concatenated in a particular order. This temporal patterning is critical to interpreting the stimulus. To explore the capacity of mice and humans to learn tactile sequences, we developed a task in which subjects had to recognise a continuous modulated noise sequence delivered to whiskers or fingertips, defined by its temporal patterning over hundreds of milliseconds. GO and NO-GO sequences differed only in that the order of their constituent noise modulation segments was temporally scrambled. Both mice and humans efficiently learned tactile sequences. Mouse sequence recognition depended on detecting transitions in noise amplitude; animals could base their decision on the earliest information available. Humans appeared to use additional cues, including the duration of noise modulation segments.

The detection of mild cognitive impairment (MCI), the transitional stage between normal cognitive changes of aging and the cognitive decline caused by AD, is of paramount clinical importance, since MCI patients are at increased risk of progressing into AD. Electroencephalographic (EEG) alterations in the spectral content of brainwaves and connectivity at resting state have been associated with early-stage AD. Recently, cognitive event-related potentials (ERPs) have entered into the picture as an easy to perform screening test. Motivated by the recent findings about the role of cross-frequency coupling (CFC) in cognition, we introduce a relevant methodological approach for detecting MCI based on cognitive responses from a standard auditory oddball paradigm. By using the single trial signals recorded at Pz sensor and comparing the responses to target and non-target stimuli, we first demonstrate that increased CFC is associated with the cognitive task. Then, considering the dynamic character of CFC, we identify instances during which the coupling between particular pairs of brainwave frequencies carries sufficient information for discriminating between normal subjects and patients with MCI. In this way, we form a multiparametric signature of impaired cognition. The new composite biomarker was tested using data from a cohort that consists of 25 amnestic MCI patients and 15 age-matched controls. Standard machine-learning algorithms were employed so as to implement the binary classification task. Based on leave-one-out cross-validation, the measured classification rate was found reaching very high levels (95%). Our approach compares favorably with the traditional alternative of using the morphology of averaged ERP response to make the diagnosis and the usage of features from spectro-temporal analysis of single-trial responses. This further indicates that task-related CFC measurements can provide invaluable analytics in AD diagnosis and prognosis.

In this paper, a method to automate industrial manufacturing processes using an intelligent multi-camera system to assist a robotic arm on a production line is presented. The examined assembly procedure employs a volumetric method for the initial estimation of object's properties and an octree decomposition process to generate the path plans for the robotic arm. Initially, the object is captured by four cameras and its volumetric representation is produced. Thereafter, a quality check with its respective CAD model is performed and the final details of the 3D model are refined. An octree decomposition technique is utilized afterwards to facilitate the automatic generation of the assembly path plans and translate them to a sequence of movements for the robotic arm. The algorithm is fast, computationally simple and produces an assembly sequence that can be translated to any major robotic programming language. The proposed algorithm is assessed and preliminary experimental results are discussed.