Abstracts

Psychophysiological Monitoring of Music Performance Anxiety and Cognitive Load through Thermography and functional Near-Infrared Spectroscopy (fNIRS)

Musical performance anxiety (MPA) is a condition that affects several musicians before to or during performances, adversely impacting both performance quality and the players' mental health and wellbeing [1]. Furthermore, MPA may modulate the cognitive workload (CW), which is the mental effort necessary to execute a task efficiently. Therefore, to assist musicians in controlling anxiety, it is essential to observe their psychophysiological conditions and cerebral activity during performances using non-invasive techniques to avoid interfering with their motions. To this end, infrared thermography (IRT) may serve as an effective instrument for monitoring the psychophysiological state of musicians, while portable neuroimaging technologies, such as functional near-infrared spectroscopy (fNIRS), could be utilized to assess cerebral activity in a natural setting, such as a concert. IRT is a contactless imaging technique that measures facial skin temperature, enabling inferences about autonomic nervous system activity in relation to individual emotional states [2], while fNIRS is a scalp-based optical method that assesses functional hemodynamics associated with cortical activity by utilizing the differential absorption characteristics of oxyhemoglobin (HbO) and deoxyhemoglobin (HHb) [3]. This research aims to assess the psychophysiological state and cerebral activity of musicians during live performances. Two preliminary investigations have been conducted for this purpose. The face temperature of four musicians was measured using IRT under two situations (i.e., rehearsal and live performance) to investigate the viability of using this method to assess the psychophysiological state of artists. The face temperature was measured using the HIRA module, which comprises co-registered optical and infrared cameras [4]. The temporal temperature profile was obtained from three regions of interest (ROIs): forehead (ROI1), nose tip (ROI2), and perioral area (ROI3), using a tracking algorithm [5].

Subsequently, the skewness, kurtosis, and sample entropy were calculated on the temperature time course of the ROIs considered. Furthermore, machine learning (ML) models were used to assess the existence of stress and the equilibrium between the sympathetic and parasympathetic systems [6]. The findings indicated significant alterations in thermal measures across all regions of interest: skewness rose by around 30%, kurtosis fell by about 45%, and sample entropy diminished by almost 80%, which literature indicates may signify heightened sympathetic nervous system activity [7]. The findings are corroborated by the ML algorithms, indicating a predominance of the sympathetic system for 50% of the rehearsal and 92% of the live performance durations. Furthermore, higher stress indicators were seen throughout 6% of the practice duration and 9% of the live performances.

The second research examined the potential of using fNIRS to monitor brain activity in performers during chamber music group performances. Specifically, two musicians (2M, ages: 18 and 21 years, saxophonist and clarinetist, respectively) participated in this research, and their cerebral activity was assessed during the performance of a piece at a concert. The saxophonist executed a piece including a solo improvisation, while the clarinetist performed a composition with a solo part. The Cortivision Photon cap device was used to assess brain activity, equipped with a 38-channel montage focused on the prefrontal, frontal, and motor cortices, in accordance with the 10–20 international standard.

Furthermore, four short-separation channels were included to alleviate physiological noise. The fNIRS data were first transformed into optical densities (ODs), thereafter underwent motion artifact correction, bandpass filtering (third-order Butterworth filter, 0.01–0.4 Hz), and were then turned into HbO and HHb concentrations oscillations. Due to the absence of a structured design in the experiment, the Automatic Identification of functional Events (AIDE) program [8] was used to directly ascertain event onsets and durations from the fNIRS data. The approach delivers General Linear Model (GLM) tstatistics, which were then used to evaluate changes in cortical activation over time. The t-statistics were averaged at 30-second intervals, and the variance between the first segment and the solo was calculated. The saxophone player exhibited a rise in prefrontal area activity, with the average t-statistic increasing from 1.715 to 8.483. The clarinet musician had a similar pattern in the prefrontal cortex (ranging from 2.158 to 11.149). These first findings indicated the viability of using IRT and fNIRS during performances to evaluate the psychophysiological condition of the musicians and their brain activity. This approach might enhance anxiety management and well-being among musicians, hence improvingperformance quality.

On the role of ancillary body movements in interpersonal synchronization during joint music making.

Joint music making is a statistical human universal, occurring in nearly all documented human societies (Savage et al., 2019; Savage et al., 2021; Patel et al., 2021), suggesting an underlying predisposition for collective musical behavior. However, empirical research in this domain has predominantly focused on trained musicians. In our study, we recruited 40 dyads: 20 composed of trained musicians and 20 of non-musicians. Each dyad performed songs using e-music boxes, digital instruments that allow participants to control the musical tempo of pre-registered music through cyclic hand movements (Novembre et al., 2015; Novembre, Mitsopoulos, & Keller, 2019). This setup enabled joint music making without requiring formal musical training, allowing us to investigate the role of visual information and body movement in interpersonal musical interaction. We examined whether ancillary body movements, i.e., those not required to produce music, facilitate interpersonal synchronization and whether this effect is modulated by musical expertise. Participants were asked to synchronize their musical output with their partner across different songs, either with or without visual access to the partner (but never to their instrumental movements). Instrumental performance was encoded as continuous oscillatory signals, enabling precise computation of synchrony between the two e-music boxes. Additionally, we recorded full-body kinematics, eye-tracking, and electroencephalography (EEG) from both participants simultaneously.

Our results show that visibility of the partner enhances instrumental synchronization precision regardless of musical training, suggesting that information retrieved from partners’ ancillary movements supports coordination. We successfully separated instrumental and ancillary body movements from 3D full-body kinematics using Principal Component Analysis. We observed that all movements exhibited two main periodicities: one matched to the musical bar, and another matched to the musical beat, and the latter was only present in the ancillary movements. This beatlevel periodicity was significantly stronger when participants could see their partner in two specific ancillary movements: upper body anteroposterior sway and head bob. Complementary, we found that partner visibility enhanced interpersonal synchronization across different homologous and heterologous ancillary movements (e.g., upper body anteroposterior sway, head bob, lateral sway, and left hand movement), specifically at frequencies aligned with the musical beat, but not at the musical bar. Both effects on periodicity and interpersonal synchronization were observed regardless of musical expertise.

To further investigate whether these ancillary movements underlie musical synchronization, we conducted a parallel mediation analysis including two predictors: (1) individual periodicity of ancillary movements, and (2) interpersonal synchronization of ancillary movements, both at the musical beat. While individual periodicity was significantly enhanced by visual access, it did not mediate musical synchronization. In contrast, interpersonal synchronization of ancillary movements mediated the effect of partner visibility on musical synchronization. This suggests that it is not merely the rhythmicity of individual movement that facilitates joint performance, but the dynamic synchrony between partners’ bodies.

These findings highlight the importance of body movement in musical interaction and suggest that visual information about a partner’s ancillary movements plays a functional role in interpersonalmusical synchrony. Notably, this effect was observed across both musicians and non-musicians, indicating that the capacity to extract coordination-relevant cues from body movement is not contingent on formal training. Ongoing EEG analyses aim to uncover the neural mechanisms that support this embodied coupling, particularly how visual perception of movement modulates sensorimotor integration during joint action.

A musical exploration of consciousness: Mind wandering, creativity, and brain dynamics

This presentation investigates the bidirectional relationship between consciousness and creative cognition through the lens of musical improvisation. Across two literature reviews and three empirical studies, music is shown to provide both a unique window into consciousness and a domain uniquely illuminated by consciousness research.

Literature Review 1 and Literature Review 2 establish the theoretical foundation, arguing that consciousness research must extend beyond dramatic clinical alterations to embrace subtle mental state fluctuations characterizing everyday cognition, with music emerging as an ideal model system, offering natural modulation of altered states, privileged access to the consciousness-behaviour relationships, and ecological validity unmatched by traditional laboratory paradigms.

Study 1 and Study 2 illustrate these very points by employing high-resolution experience sampling during live jazz improvisation to probe mental states and their functional consequences on ongoing musical creativity. Study 1 provided proof-of-concept (n = 9) using a simplified binary classification (task-focused attention vs mind-wandering). Contrary to paradigms positioning mind-wandering as detrimental, preliminary evidence showed it can instead facilitate creative cognition in ecological settings. Study 2 replicated and extended these findings (n = 52) using refined taxonomy distinguishing multiple on-task (task-focused attention, task-related interference) and off-task states (mind-wandering, mind-blanking).

Results revealed meaningful functional heterogeneity: mind-wandering enhanced creativity, task-related interference suppressed it, and mind-blanking showed modest benefits. Expertise moderated these effects: novices derived greater creative benefit from mind-wandering despite experiencing it less frequently, suggesting spontaneous thought compensates for not-yetoptimized creative processes.

Study 3 provided the first causal evidence of distinct cortical dynamics during musical improvisation by combining transcranial magnetic stimulation (TMS) with high-density electroencephalography (EEG) in professional jazz pianists (n = 3) during active musical improvisation, sight-reading or rest. Representing the first successful application to complex motor behaviour, results revealed that improvisation operates through neural efficiency rather than complexity: reduced local mean field power, decreased phase-locking, preserved but gain-modulated early components, and reduced global perturbational complexity.

Three major theoretical contributions emerge. First, rather than failures of cognitive control, spontaneous thought may signal adaptive cognitive flexibility in naturalistic cognition; optimal performance at lower skill levels emerges not from unwavering focus but from flexible navigation between different mental states. Second, findings expand the Dynamic Framework of Thought by distinguishing functionally distinct subtypes: two forms of mind-blanking (sleep-like vs absorptive), and within the previously undifferentiated goal-directed thought family, task-focused attention as its standard manifestation and task-related interference as an over-constrained variant. Third, creative states stem from efficient, streamlined cortical processing, not complex, unconstrained cognition. As consciousness research continues to grapple with the subjective, dynamic nature of experience, musical paradigms offer a path forward: ecologically valid, experimentally tractable, and rich in their capacity to reveal essential properties of human experience.

Embodiment indicators of music expertise: psychophysiological and biomechanical markers of music skill acquisition.

Music performance has been described as a complex sensorimotor activity that involves acquiring cognitive, motor, and affective skills. Previous theoretical work argues that this acquisition process is a gradual process of embodiment, during which the instrument becomes integrated into the musician’s body and becomes experienced as a natural extension of the self. Building on this framework, we argue that the embodiment process becomes observable in a set of what we call embodiment indicators. These are biomechanical and psychophysiological patterns that reflect how the instrument impacts the body through the learning process.

Despite extensive research on the neural and physiological basis of music performance, little attention has been given to how these indicators evolve with skill acquisition. To begin addressing this gap, we conducted a scoping review on studies that use psychophysiological and biomechanical methods to investigate music skill acquisition, with the aim of identifying embodiment markers that manifest at different levels of expertise. By focusing on measurable indicators, her work aims to identify concrete markers that capture the development of embodied expertise in music skill acquisition and performance.

SynchMuse: a novel paradigm to study embodied coordination in musical ensembles using virtual avatars

Ensemble music performance provides a rich testbed for examining how humans achieve and maintain rhythmic synchrony in real time(1). Yet much of the available literature still concentrates on dyads, simple rhythms, or single-modality data, leaving open fundamental questions about how larger groups coordinate, how subgroup structures shape synchrony during polyrhythmic performance, and how these dynamics can be captured and modelled with sufficient precision(2–5). At the same time, emerging technologies in extended reality (XR) and embodied virtual agents (EVAs) introduce new methodological possibilities that raise further questions: How can multimodal group interaction data (e.g., timing, gesture, posture) be acquired, aligned, and analysed to develop computational models? And how can such models be embodied in EVAs that behave in musically plausible and adaptive ways?

SynchMuse, a bilateral project between Ghent University (IPEM) and McGill University (SPL, IDMIL/CIRMMT), aims to address these questions by first developing computational methods for analysing and modelling group synchrony, and then embedding these models in an EVA, capable of interacting with human performers in real time. These goals are organised into three stages: (1) designing a multimodal experimental protocol for bilateral data collection of timing and movement in quartets;

(2) developing computational models to estimate group-level coordination, fitted to the collected data; and

(3) building an XR framework in which these models drive an embodied virtual musician capable of real-time synchronisation with an ensemble.

Methods: Prior to data collection, we designed a controlled multimodal protocol to capture rhythmic coordination in quartet tapping tasks, aimed at revealing the dynamics of group-level coupling and adaptation. Sixteen quartets of trained musicians were recruited—eight in Montréal and eight in Ghent. The experiment was implemented identically across the two institutions. Participants were divided into two dyads and performed continuous tapping under four rhythmic conditions: Binary (2:1), Ternary (3:1), Simple (all performers share the same subdivision), and Complex (3:2 polyrhythmic dyads). Trials contained multiple perturbation cues at random locations, after which the dyads switched rhythmic roles, generating unpredictable instability and forcing re-synchronization Data collection was carried out in parallel in Ghent and Montréal using a custom multimodal setup consisting of: low-latency tapping sensors connected to a Bela board, to capture millisecond-level onsets; upper-body Qualisys motion capture to record posture, body sway, and gesture; high-quality video for cross-validation; and subjective questionnaires probing flow, coupling, and perceived synchrony.

This multimodal setup enables detailed comparisons between timing-based and movement-based measures of coordination, which will be presented in the symposium. Preliminary analyses offer the first quantitative description of how quartets adapt when exposed to perturbations during polyrhythmic tasks. In this presentation, we will share these initial findings to advance our understanding of joint action in musical ensembles. Particular attention will be given to how subgroup dynamics shape and constrain full-group coordination, and which nonverbal communication cues are most relevant for maintaining performance stability.

By presenting this experimental protocol and the first empirical insights from the collected data, this contribution aims to offer methodological guidance for studying rhythmic coordination in musical ensembles. Taken together, the three SynchMuse stages demonstrate how multimodal sensing and dynamical modelling can be integrated into a novel paradigm for studying coordination in musical ensembles. Moreover, they lay the groundwork for investigating coordination in XR-based human–agent joint task.

Beyond tension and relaxation: Optimizing antagonistic muscular co-contraction (AMCC) for skilled, healthy piano technique

Aims: This doctoral thesis sought to develop new understanding of antagonistic muscular cocontraction (AMCC) in the context of piano playing, to aid pianists facing playing-related challenges.

Background: Playing the piano skillfully is an exceptionally complex task. Pianists are confronted by considerable obstacles when developing and maintaining their playing. Debate about the physiological mechanics of proper piano technique has continued across pedagogical and scientific fora for centuries, including disagreement over AMCC. A scientifically sound and pedagogically viable understanding of proper piano technique is overdue — which must clearly account for muscle activity, including AMCC’s effects and proper role.

Methods: This thesis justifies its novel, integrative methodology by determining that prior scientific and practical inquiry has been unable to properly comprehend the complex phenomenon of AMCC, partly due to persistent theory-practice gaps. The thesis leads this novel approach by adopting and harmonizing previously untried methods. Chapter II’s scoping review establishes a science-based theoretical model of AMCC, which is applied and expanded in Chapter III’s systematic critical review of pedagogy. Chapter IV then generates a practice-led model and integrates this with the prior expanded model, culminating in a unified theory-practice model of AMCC in piano playing. Chapter V observes pianists’ AMCC via surface electromyography (sEMG), empirically validating key areas of this unified model.

Results: The thesis’s unified model thoroughly and verifiably characterizes AMCC’s nature and effects on piano playing. AMCC is a fundamental contributor to human movement with broad and unique consequences, including a dualistic capacity for major positive and negative effects both on sensorimotor task performance and musical expression — but prior AMCC research and pedagogy are fragmentary and often flawed, frustrating prolonged efforts at understanding and harnessing its potential.

Conclusion: This thesis synthesizes multifarious perspectives on AMCC, arriving at a unified model of AMCC’s nature and effects on piano performance. Skilled, healthy piano playing requires considerable AMCC, due to its capacity for substantial, transcendent, and uniquely beneficial effects on the exceptional challenges of piano performance — but this AMCC must be implemented dynamically and organically within the broader context and expressive function of musical performance, otherwise quickly becoming harmful rather than facilitative. The thesis Ashkenazi 3concludes that pianists can and should train, use, and maintain AMCC, which itself varies significantly between and among individuals — thereby illuminating clear avenues for future pedagogical transformation. These findings constitute a compelling resolution to centuries of debate, misunderstanding, and inconsistency regarding AMCC.

Enacting Social Intention in Ensemble Performance: A Multimodal Framework for Analyzing Embodied Co-regulation.

Human musical interaction provides a rich model for understanding how social meaning emerges through embodied co-regulation (Schiavio & Høffding, 2015; Leman & Maes, 2017). Existing ensemble studies have primarily focused on sensory and temporal mechanisms of coordination, emphasizing prediction, entrainment, and perceptual-motor coupling (Keller, 2014; Novembre & Keller, 2014), while the role of social and affective intention in shaping these dynamics remains less systematically examined (Krueger, 2013; Schiavio & De Jaegher, 2017). Yet performers’ willingness to engage or withdraw fundamentally alters how they attend, move, and adapt to one another, suggesting that musical interaction and coordination should be understood not only in terms of perceptual coupling but also as an enacted social process (De Jaegher & Di Paolo, 2007; Schiavio & Høffding, 2015).

The AURA project develops a multimodal, psychologically grounded framework to explore this phenomenon in a systematic way. The first stage of AURA introduces a controlled paradigm which focuses on human-human piano duets to examine how instructed social-affective states influence coordination and expressivity. Pianist pairs perform a short, cue-based composition under instructions that modulate their social-affective orientation, encouraging more engaged or more withdrawn forms of interaction. The piece, specially written for this paradigm, includes controlled musical cues (entries, pauses, expressive transitions) that require real-time negotiation and provide repeated and systematic reference points for analysis. This design enables direct comparison of behavioral patterns across conditions while maintaining the authenticity of musical performance.

Behavioral analysis is structured around two complementary domains: interactional behavior, referring to coordination mechanisms such as mutual attention, anticipatory gestures, and temporal alignment, and expressive behavior, referring to bodily and facial expressions that reflect emotional and interpretive intent. Quantitative multimodal data (movement, facial expression, and musical timing) are combined with qualitative and psychological measures, including performer self-reports and expert evaluations of engagement, synchrony, and expressivity. This multimodal, event-locked analysis, in which the ‘events’ correspond to structural cues embedded in the score, links musical structure with embodied and affective dimensions of interaction; it reveals how instructed social-affective states modulate coordination and expressivity at specific musical moments. By integrating behavioral, musical, and psychological data, the study develops a shared vocabulary of embodiment in musical interaction that describes and categorizes expressive gestures and coordination patterns, advancing understanding of how social engagement and musical meaning are enacted through the body.

Within the broader AURA framework, these findings provide the empirical and psychological foundation for later studies that will use expressive virtual agents to simulate human interaction patterns. While later phases will explore whether comparable interaction patterns can emerge in human-avatar performance, the present work focuses on the enactment phase, examining how social-affective states are embodied and co-regulated between human performers. Together, these efforts establish a replicable methodological model for investigating embodied interaction in music and other social contexts, bridging embodied music cognition, psychology, and multimodal analysis. The presentation will outline the theoretical background and cue-based, multimodal method, and discuss initial implications for analyzing embodied co-regulation.

Investigating human learning and control in piano performances.

Skilled pianists demonstrate fluent and efficient sound-producing motion, which is usually acquired over many years of practice. In the context of a longitudinal study of piano performance rehearsal, we are collecting a large multimodal dataset consisting of at-home recordings of multiple pianists working through different musical pieces at different stages of their musical realization [1]. This provides us with the opportunity to observe in an ecologically-valid way the progressions from the initial stages of preparation to the final performances of musical pieces.

The work described here will focus mainly on the study of motor control skill acquisition by measuring the kinematics and dynamics of the pianists’ sound-producing movements. We aim to assess both inter- and intra-pianist differences in such measurements and follow their development through time in order to better understand their learning and control strategies.

We utilize a combination of music information retrieval (MIR), deep learning (DL), and artificial intelligence (AI) methods, in combination with the collected audio, MIDI, video, and textual data to conduct our analysis.

Initial work has utilized MIR and MIDI in connection with rehearsal structure analysis and mistake identification [2], which provides us with a learning assessment and segmentation framework. We have developed methods to capture and analyze the kinematics of the hand and finger joints with the use of automatic hand pose estimation from video recordings [3], and we will extend this to the trunk-shoulder-arm-hand kinematic chain and also to the estimation of movement dynamics in a non-invasive way, which would not interfere with the ecological setting of the rehearsals. We conducted an experimental study, which showed measurable differences between more- and less-experienced pianists in the smoothness and quantity of motion of their fingers, as well as intra-pianist differences under different tempo demands (work currently under peer-review).

We are also working on the development of new DL and AI methods for the computational understanding and analysis of performed music, e.g. automatic music transcription from silent videos [4]. We aim to connect such measurements with other physiological (EMG, ECG, skin temperature) and psychological (self-conducted questionnaires) markers, in order to conduct more in-depth analysis and find correlations between different modalities and between objective observations and subjective goal-setting.