The processing and learning of temporal patterns: from behaviour to computational models

Being able to successfully and adaptively interact with our environment often requires us to process and learn various forms of temporal regularities, involving different types of temporal features and structures. In many cases the presence of such regularities makes us experience a series of events as a temporal pattern. We often identify a series of events as a pattern based on its temporal properties, like the duration of the events, their order and temporal relationship. Temporal patterns feature a multitude of human sensorimotor experiences: musical rhythms, predictions and expectations, complex motor actions, speech perception and productions. Likewise, the interest in temporal patterns and their investigation have involved several research fields and perspectives. Studies from all these fields have shown humans’ remarkable abilities in perceiving and learning different types of temporal regularities and patterns, in using them to build expectations, orient attentional resources and organize motor actions. Despite the growing body of knowledge on the basic mechanisms supporting these abilities, there are still many unanswered questions concerning how humans process and learn temporal patterns. A small portion of these open questions is the focus of the current PhD thesis. Given the broad nature of the topic, the current work adopts a “crossroad” perspective that combines theoretical and analytical tools from timing and sequence processing research. The first part of the work (chapter 2) focuses on the processing of different levels of regularity of a temporal pattern. Specifically, we asked how the reproduction of a temporal pattern is affected by a violation within its elements (i.e. local violation) and in its overall structure (i.e. global violation). To answer this question, we adapted a paradigm originally developed in sequence processing studies to the temporal domain, using a Synchronization-Continuation task. This design was employed in three behavioral experiments, in which we tested the effects of structure complexity and sensory modality on the behavioral signatures associated with local and global violations. Analysis of behavioral data was carried in the frequency domain to obtain a measure of tapping precision, and with Procrustes technique to investigate the internal consistency of the tapping. Results from frequency-domain analysis showed that local violations of a temporal pattern always affected the precision of its reproduction, whereas global violations were detrimental when using visual and complex patterns. Analysis of participants’ performances in the time-domain using Procrustes technique showed that in all three experiments the detrimental effects observed in the frequency-domain were partially explained by a systematic tendency of participants to reproduce rescaled versions of the sequences. This last result suggests that the accuracy in reproducing a temporal pattern, does not reflect the ability to acquire and reproduce its global structure. The second part of the work focuses on the learning of temporal patterns, addressing the topic from two different perspectives. In the first study (chapter 3) we asked how humans learn to switch from a familiar to a novel temporal structure and how this switch is modulated by trial predictability. This question was tackled employing a novel experimental design in which we parametrically changed the probability of occurrence of two mirroring patterns across trials. This design was applied to a synchronization task presenting the patterns within an auditory stream. Analysis of behavioral performance combined standard tapping measures and a model-based approach. The general purpose of the model-based analysis was to describe and estimate the parameters underlying the learning transition between the two temporal structures, as expressed in subjects’ tapping. Results showed that the models implemented were able to retrieve the probabilities in which the two patterns were reproduced by our participants, reflecting the learning transition they experienced during the task. The combination of standard measures and models’ results allowed to identify different types of performance and learning strategies among participants. A group of “fast learners” learned to synchronize with the first unknown pattern almost immediately and showed higher sensitivity to the switch in pattern across trials. On the contrary the “slow learners” took more time to both learn to synchronize with a new pattern and switch to a different one. In the second study (chapter 4) we asked whether and how humans extract and learn low- and high-level statistical properties underlying non-deterministic sequences of temporal intervals. To this aim we designed three generative Markov processes, each defined by a set of statistical properties. Based on these processes we created multiple sequences and presented them to our participants in a delayed reproduction task. Model-based and data-driven analysis were applied to behavioral performances to investigate the statistical properties learned by our participants. Overall results showed that participants were able to learn the generative processes of the sequences they experienced, but also that additional statistics interacted with those designed in our stimuli, namely a central tendency effect and a shrinkage of the duration categories’ space. The work of this thesis gives insights on important aspects concerning the processing and learning of temporal patterns. A first insight is that the reproduction of a temporal pattern seems to require an interplay between individual constitutive durations and overall structure. These two levels are differently weighted according to the complexity of the pattern and the sensory modality in which it is experienced. The extent to which the weights of individual durations and overall structure depend on the “status” of the pattern (rhythmic or aperiodic) and the type of temporal task is not fully clear. Another insight comes from the role of the predictability of the environment in shaping the learning of temporal patterns. The level of predictability not only influences the ability to learn new temporal structures, but also interacts with subject-specific strategies employed to perform the task. When learning the statistical properties of non-deterministic sequences of durations, instead, the level of predictability modulates the interaction between statistics of the stimuli and low- and high-level perceptual priors.