Finn Upham

• Upham, Finn & Rosas, Fernando E. (2025). When and How the Audience Moves. Music & Science. 8, p. 1¨C24. doi: 10.1177/20592043251398280. Full text in Research Archive Show summary

  Audience motion is a defining component of the face-to-face concert experience. Motion is also a common component of individual audience members¡¯ reactions and actions to live performances. This article investigates patterns in chest-mounted accelerometer measurements from audience members at a chamber music concert experiment. The aim is to contextualize their behavior during music performance using a sequence of exploratory analyses. Combining insight from subjective reports of audience motion; musical, environmental, and biological frequencies; and similarity of motion timing with neighbors or more distant participants, both in-hall audience participants and live-stream viewers show two distinct music¨Cmotion patterns: either unobtrusive dynamic stilling or noticeable tempo-matched bouncing. Audience motion outside of music, during applause, speeches, or survey writing, also shows distinct average quantities of motion and degrees of shared information between seat neighbors, prompting a discussion on the constrained opportunities for audience action and social interaction at live classical concerts. Although exploratory, these extensive analyses provide a useful foundation for future work to further understand audience behaviors in concert music settings and beyond.

• Upham, Finn & Burnim, Kayla (2025). ACCELEROMETER-BASED SYNCHRONISATION PROTOCOL FOR MUSIC-ALIGNED MEASUREMENTS OF DISTRIBUTED DATALOGGING WEARABLES. In Frank, Matthias & Vallejo, Sofia (Ed.), Proceedings of the Sound and Music Computing Conference 2025. SMC Network. ISSN 9783200106420. p. 415¨C420. doi: 10.5281/ZENODO.15838480. Full text in Research Archive Show summary

  Many commercial wearable biosensors can be set to record (datalogging) cardiac activity, motion, respiration, and other signals in devices easily worn during group musical activities. However, despite offering research-relevant sample rates and exportable data formats, these commercial units are not designed for high accuracy synchronisation to external clocks. Anticipated uses like solo exercise and sleep monitoring do not require inter-device alignment below a few seconds, while musical activities feature discernible events within 200 ms and action alignment between participants under 10 ms. In the absence of a mechanism to digitally communicate timing triggers across datalogging devices, we have developed a synchronisation protocol to embed audio-aligned cues in active accelerometers to allow reassignment of timestamps after recording. This process substantially improves inter-unit signal alignment, correcting initialisation time offsets and clock drift to within accelerometer sample rate in recordings spanning 3 to 10 hours. Implemented in 15 recording sessions over 5 experiments, involving 9 to 95 units of various datalogging wearables, the resultant datasets support analysis of inter-participant coordination across signals and, with audio-based dynamic time warping, relatively high precision comparisons of sensor measurements to repeated musical events, all outside of laboratory conditions.

• Lartillot, Olivier; Swarbrick, Dana; Upham, Finn & Cancino-Chac¨®n, Carlos Eduardo (2025). Video Visualization of a String Quartet Performance of a Bach Fugue: Design and Subjective Evaluation. Music & Science. 8. doi: 10.1177/20592043251352299. Full text in Research Archive Show summary

  Visualizing music¡ªthrough music notation, analytical representations, or music videos¡ªmight potentially boost the appreciation of music in all its richness. The purpose of this study was to design and test a visualization strategy aimed at explicating to a large audience with diverse backgrounds¡ªespecially novices¡ªthe multifaceted beauty of the final Contrapunctus in J.S. Bach's The Art of Fugue, performed by the Danish String Quartet. At the surface level of the musical structure, the rich fluctuation of pitch shaped by each musician was depicted in the form of undulating pitch curves. At a deeper structural level, the repetition of pitch curves, distinctive of fugues, was highlighted through vertical alignment¡ªinspired by a technique called paradigmatic analysis, originating from anthropology and music semiology. The visualization was initially prototyped in the form of a real-time technology as part of the MusicLab Copenhagen research concert. The concert audience focused on the performance itself, and did not pay much attention to, nor appreciate, the visualization. To evaluate more thoroughly the potential of the visualization, participants with varied musical expertise and taste were invited to listen to a recorded performance of the piece and watch the visualization on their own computer. A large majority reported that they felt they understood the visualization, around half of them felt that it enhanced their musical understanding, and a small group felt that it helped them to better appreciate the music.

• Haswell-Martin, Remy; Upham, Finn; H?ffding, Simon & Nielsen, Nanette (2025). Embodied, Exploratory Listening in the Concert Hall. Behavioral Sciences. 15(5). doi: 10.3390/bs15050710. Full text in Research Archive Show summary

  Live music can afford novel, transformative aesthetic interactions for individual audience members. Nevertheless, concert research tends to focus on shared experience. In this paper we offer an account of exploratory listening that foregrounds embodied¨Cenactive engagement and affective resonance through close analysis of the music, physiological measurements, and reflections from interviews. Our analysis centres on data collected from two musician audience members about one specific piece out of a larger interdisciplinary project involving concerts given by the Stavanger Symphony Orchestra and The Norwegian Radio Orchestra in March and June of 2024. Through the combination of in-depth phenomenological interviews with musically skilled audience members and measurements of breathing and body motion, we explore aesthetic enactment beyond common patterns of ¡®synchronised¡¯ response, focusing on audience members¡¯ experiences of Harald S?verud¡¯s ¡®Kjempevisesl?tten¡¯ (The Ballad of Revolt) (1943). We find forms of absorbed, both imaginative and embodied involvement, of listeners enacting meaningful contact with, and pathways through, the music that in some ways corroborate crowd patterns but also reveal exploratory expertise and idiosyncratic affective orientations.

• H?ffding, Simon; Bergstr?m, Rebecca Josefine Five; Bishop, Laura; Bravo, Pedro Pablo Lucas; Burnim, Kayla & Cancino-Chac¨®n, Carlos Eduardo [Show all 28 contributors for this article] (2025). Introducing the MusicLab Copenhagen Dataset. Music & Science. 8. doi: 10.1177/20592043241303288. Full text in Research Archive Show summary

  MusicLab Copenhagen was a unique research concert featuring the world-renowned Danish String Quartet in a naturalistic setting. The audience was split between one group physically located in the hall, another group listening to a radio broadcast, and a third group watching a live stream. Qualitative and quantitative data were captured from both musicians and audiences, resulting in a comprehensive dataset that can be used to address many research questions. This document introduces the dataset, explains its structure, and reflects on the related data collection, storing, publishing, and archiving processes.

• Upham, Finn; H?ffding, Simon & Rosas, Fernando E. (2024). The Stilling Response: From Musical Silence to Audience Stillness. Music & Science. 7, p. 1¨C14. doi: 10.1177/20592043241233422. Full text in Research Archive
• Upham, Finn; Lee, Jin Ha & Park, So Yeon (2023). Audience reconstructed: social media interaction by BTS fans during live stream concerts. Frontiers in Psychology. 15, p. 1¨C17. doi: 10.3389/fpsyg.2024.1214930. Full text in Research Archive Show summary

  COVID-19-motivated social distancing made online concerts common practice in 2020 and 2021, with millions logging into streaming sites to see their favorite artists perform in realtime. For some fans, watching alone at home may have been enough, but concert-concurrent surges of social media activity suggest many virtual performance attendees are doing more. To understand why fans would turn their attention from these precious performance streams to social media, we explored Twitter engagement during four live streamed concerts performed by the Kpop group BTS in 2021. In public Tweets sampled by either concert hashtag or a predefined stream of users and keywords, we evaluated patterns in posting rates in relation to concert program events and investigated the content patterns in 1,200 Tweets sampled from four ranges of popularity (number of Retweets during the concert). Across concerts, short "Shout" Tweets surged at the start of songs, while the rate of retweets often fell during musical performances and shot up when BTS was off stage. Content analysis on the subsample found the materials most widely shared were informational or featured concert visuals, mimicking how fans use their phones at in-person concerts. Most original posts received few Retweets and were more personal and expressive of admiration for the performers. Comparison between the samples (concert hashtag vs. stream) also suggests users were strategic in using or omitting official concert hashtags with the strongest differences in the most widely disseminated content. Postings on Twitter during these performances seemed principally directed to fellow fans and audience members, by individuals choosing to share their own excitement and check in with others. By leveraging their existing social media networks, these concert attendees constructed a collective and interactive concert space, connecting with friends and strangers in the crowd and helping each other capture a richer experience than any broadcasting platform currently supports.

• Riaz, Maham; Upham, Finn; Burnim, Kayla; Bishop, Laura & Jensenius, Alexander Refsum (2023). Comparing inertial motion sensors for capturing human micromotion, Proceedings of the Sound and Music Computing Conference 2023. SMC Network . ISSN 9789152773727. doi: 10.5281/zenodo.8316051. Full text in Research Archive Show summary

  The paper presents a study of the noise level of accelerometer data from a mobile phone compared to three commercially available IMU-based devices (AX3, Equivital, and Movesense) and a marker-based infrared motion capture system (Qualisys). The sensors are compared in static positions and for measuring human micromotion, with larger motion sequences as reference. The measurements show that all but one of the IMU-based devices capture motion with an accuracy and precision that is far below human micromotion. However, their data and representations differ, so care should be taken when comparing data between devices.

• Swarbrick, Dana; Upham, Finn; Erdem, Cagri; Jensenius, Alexander Refsum & Vuoskoski, Jonna Katariina (2022). Measuring Virtual Audiences with The MusicLab App: Proof of Concept. In Michon, Romain; Pottier, Laurent & Orlarey, Yann (Ed.), Proceedings of the 19th Sound and Music Computing Conference. SMC Network. ISSN 9782958412609. p. 532¨C539. doi: 10.5281/zenodo.6798290. Full text in Research Archive Show summary

  We present a proof of concept by using the mobile application MusicLab to measure motion during a livestreamed concert and examining its relation to musical features. With the MusicLab App, participants¡¯ own smartphones¡¯ inertial measurement unit (IMU) sensors can be leveraged to record their motion and their subjective experiences collected through survey responses. The MusicLab Lockdown Rave was an Algorave (live-coded dance music) livestreamed concert featuring prolific performers Renick Bell and Khoparzi. They livestreamed for an international audience who wore their smartphones with the MusicLab App while they listened/danced to the performances. From their acceleration, we computed quantity of motion and compared it to musical features that have previously been associated with music-related motion, namely pulse clarity and low and high spectral flux. By encountering challenges and implementing improvements, the MusicLab App has become a useful tool for researching music-related motion.

View all works in NVA

• Arnim, Hugh Alexander von; Christodoulou, Anna-Maria; Burnim, Kayla; Upham, Finn; Kelkar, Tejaswinee & Jensenius, Alexander Refsum (2025). LightHearted¡ªA Framework for Mapping ECG Signals to Light Parameters in Performing Arts. Full text in Research Archive
• Upham, Finn & Burnim, Kayla (2025). ACCELEROMETER-BASED SYNCHRONISATION PROTOCOL FOR MUSIC-ALIGNED MEASUREMENTS OF DISTRIBUTED DATALOGGING WEARABLES. doi: 10.5281/zenodo.15838480. Full text in Research Archive Show summary

  Many commercial wearable biosensors can be set to record (datalogging) cardiac activity, motion, respiration, and other signals in devices easily worn during group musical activities. However, despite offering research-relevant sample rates and exportable data formats, these commercial units are not designed for high accuracy synchronisation to external clocks. Anticipated uses like solo exercise and sleep monitoring do not require inter-device alignment below a few seconds, while musical activities feature discernible events within 200 ms and action alignment between participants under 10 ms. In the absence of a mechanism to digitally communicate timing triggers across datalogging devices, we have developed a synchronisation protocol to embed audio-aligned cues in active accelerometers to allow reassignment of timestamps after recording. This process substantially improves inter-unit signal alignment, correcting initialisation time offsets and clock drift to within accelerometer sample rate in recordings spanning 3 to 10 hours. Implemented in 15 recording sessions over 5 experiments, involving 9 to 95 units of various datalogging wearables, the resultant datasets support analysis of inter-participant coordination across signals and, with audio-based dynamic time warping, relatively high precision comparisons of sensor measurements to repeated musical events, all outside of laboratory conditions.

• Upham, Finn (2025). Workshop: Coherence and Cause - Exploratory analysis strategies for physiological measurements from live concert participants. Full text in Research Archive Show summary

  Concert research involves studying humans engaged in very complex activities: highly predictable but constantly changing stimuli experienced through multiple sensory modalities and socio-cultural constraints. Physiological measurements of audience members and musicians during such activities reflect that richness with substantial differences between participants, with substantial ambiguity around the contributions of the shared musical activity on their respective heart rates, respiratory sequences, and movement timing. Some analysis strategies treat this richness as noise around more obvious correlates identifiable in laboratory conditions (ex: loudness, listener arousal, and heart rate). Others invoke complex data-agnostic assessments of shared information to demonstrate the existence of synchronicity with little opportunity to investigate how intervals of coherence emerge, whether across a crowd or between performers and audience. This workshop presents a data-centric approach designed to facilitate the discovery and evaluation of causal connections between musical information and physiological events during concerts. With examples from audience motion and performer respiration, Finn will demonstrate this process of exploration, identification, and probabilistic evaluation and then share some data from past studies to be explored by attendees and discussed as a group. Key issues discussed will include: Assessing the limits of a dataset, whether it can provide enough information to test for the phenomenon suggested. Controlling the analysis sequence through identification and evaluation. Projecting the consequences of the identified phenomenon and performing orthogonal tests on the same or related datasets. Conditions for dropping the thread when the evidence doesn't hold up, and making the most of analysis mistakes. Hypothesis-lead controlled experimental work may be the gold standard for building scientific knowledge, but there is more to discover than that which we already know to look for. Exploratory approaches that are sensitive to both the mechanisms of music production as well as the characteristics of measured physiological signals can expose patterns with more explanatory power than large-scale evaluations of coherence. Potential exploration datasets include: - respiratory-cardiac interactions in brass players - laughter and concert audience motion - Applause textures, synchronous vs independence

• Upham, Finn (2025). Respiratory timing in orchestral performance. Full text in Research Archive Show summary

  Breathing is a necessary physiological function during music performance, and the timing of respiration by wind players and singers are reliable constrained by their performance actions. The relevance of respiration timing to other instrumentalists and across large ensembles has not be investigated in detail. One study on solo pianists found little reliability, while many studies of string quartets and other chamber groups often have not bother to monitor performers¡¯ breathing. The Bodies in Concert experiments measured orchestral musicians breathing, cardiac activity, and motion during repeated performances in three concert series. Looking closely at the timing of respiratory sequences captured for a set of annotated pieces (6 pieces, 5-8 performances, 30 - 55 musicians per ensemble), we consider specifically the coincidence of inspiration and expiration onsets between performers in each section of the orchestra and within performers between performances with audio-based dynamic time warping. These orchestral musicians¡¯ breathing show consistent timing beyond the most expected cases. Performers varied in their intra-performance reliability, averaging 20-50% consistency while playing (depending on the section) and 10-15% while tacet. Between players, expiration onset coincidence rates were also significantly higher that chance or alternative conditions for all string sections as well as brass and winds. Patterns within string sections over repeated performances highlight passages for which these players reliably recruit their respiratory system. However, these moments do not appear sufficient to fully explain the exceptional degree of respiratory coordination measured in each performance. The respiratory patterns captured show substantial disruptions allowing alignment with musical action rather than coincidental phasing between steady oscillatory systems, even in non winds. For many of these musicians, breathing looks to be an important if not always essential component of precise musical performance.

• Upham, Finn (2024). Heart Rate consistency and Heart Rate Variability constraints in orchestral musicians across performances. Full text in Research Archive
• Upham, Finn (2023). Making the best of it: Constraints and opportunities in studying complex human musicality. Full text in Research Archive Show summary

  A talk for the Oslo Centre for Biostatistics and Epidemiology on exploratory data analysis with examples of strategies and challenges that arise in studying measurements from real world activities like orchestral concert performances. Presents preliminary findings on audience motion patterns and orchestra respiration, and presents the advantages of studying expert bodies in action.

• Upham, Finn (2023). Using Metrically-entrained Tapping to Align Mobile phone sensor measurements from In-person and Livestream Concert Attendees. Full text in Research Archive Show summary

  Music is often made and enjoyed in large groups, but simultaneously capturing measurements from dozens or hundreds of people is technically difficult. When measurements are not constrained to wired or continuous connected wireless systems, we can record much bigger groups, potentially taking advantage of the wearable sensors in our phones, watches, and more dedicated devices. However, aligning measurements captured by independent devices is not always possible, particularly to a precision relevant for music research. Phone clocks differ and update sporadically, wearable device clocks drift, and for online broadcast performances, exposure times can vary by tens of seconds across the remote audience. Many measurement devices that are not open to digital synchronisation triggers still include accelerometers; with a suitable protocol, participant movement can be used to imbed synchronisation cues in accelerometry measurements for alignment regardless of clock times. In this paper, we present a tapping synchronisation protocol that has been used to align measurements from phones worn by audience members and a variety sensors worn by a symphony orchestra. Alignment with the embedded cues demonstrate the necessity of such a protocol, correcting offsets of more than 700 ms for devices supposedly initialised with the same computer clock, and over 10 s for online audience participants. Audience tapping performance improved cell phone measurement alignment to a median of 100 ms offset, and professional musicians tappings improved alignment precision to around 40 ms. While the temporal precision achieved with entrained tapping is not quite good enough for some types of analyses, this improvement over uncorrected measurements opens a new range of group coordination measurement and analysis options.

• Upham, Finn (2023). Breathing Together in Music, a RESPY Workshop. Full text in Research Archive Show summary

  Respiration is a subtle but inescapable element of real time musical experiences, sometimes casually accompanying whatever we are hearing, other times directly involved in the actions of sound generation. This workshop explores respiratory coordination in music listeners and ensemble musicians with respy, a new python library for evaluating respiration information from single belt chest stretch recordings. Following an introduction to the human respiratory system and breathing in music, the workshop demonstrates how the respy algorithms evaluate phase and breath type, and presents statistical tools for assessing shared information in these features of people listening to or making music together. Rather than only use aggregate statistics such as respiration rate, respy aims to elevate the details of the respiratory sequence to facilitate our exploration of how breathing is involved in musical experiences, second-by-second. Measurable coordination of the respiratory system to musical activities challenges our expectations for interacting oscillatory systems. This session will conclude with a discussion on the different categories of relationships possible between people breathing together in music.

• Upham, Finn & Oddekalv, Kjell Andreas (2023). Fingers and Tongues: Appreciating Rap Flows through Proprioceptive Interaction in Rhythm Hive. Full text in Research Archive Show summary

  Rhythm games have been studied for their potential to develop interest in music making (Cassidy and Paisley, 2013) and transferable musicianship skills (Richardson and Kim, 2011), but how might they influence players appreciation for specific musical works? Proprioceptive interaction, a concept by game designer Matt Bloch (Miller, 2017), refers to changes in a game player's perception of music as they practice specific movements to it. By drawing attention to coincidental sounds, players can develop their hearing and appreciation for nuances of production and performance. Many fans of rap enjoy performances in languages they do not speak themselves. Without specific language skills, expertise in rap performance, and/or time to learn lyrics phonetically, their experience of a rap flow is hampered by an inability to imitate and imagine the generative action of performance. Rhythm Hive is a mobile rhythm game based on the music of BTS, Enhyphen, and TXT, Kpop groups with substantial followings outside of Korea. Game play presents players with finger choreographies to these groups¡¯ hit songs, tapping sequences to the vocal performances across four to seven positions in a line. For these groups¡¯ many non-rapping and non- Korean-speaking fans, playing Rhythm Hive may offer deeper understanding of performances by rappers like RM, Suga, and J-Hope. Through expert analysis of rap performance, transcriptions of game play, and reflections on the experience of playing Rhythm Hive, we consider shared structure between the prescribed finger choreographies and the rap flows they accompany. We studied rap verses from four BTS songs along side their Easy and Hard level tapping sequences (vocal versions only) to identify parallels in rhythm, segmentation, repetition, and accents. Easy mode choreographies tend to mark their relationship to rap vocals by hitting the start of lines and then articulating structure with repeated contours tapped on quarter and eighth notes. Hard mode choreographies tend to hit every rapped syllable and incorporate more gestural flourishes to mark pitch changes, ending and internal rhymes, and interesting breaks from a steady 16th note flow. Both Easy and Hard tappings sequences consistently follow the rap track when it deviates from a quantized beat. The finger choreographies of Rhythm Hive illuminate rap performances by directing and rewarding players¡¯ attention to details of flows that may otherwise be missed. Game feedback pushes players to replicate delivery microtiming, while spatial patterns underline linguistic and rhythmic structure. Hard mode tapping sequences articulate distinguishing characteristics of specific rap styles, given players tangible sensitivity to degrees of technicality and nuances of genre. While fans may be motivated to play rhythm games like Rhythm Hive out of a preexisting love of the music and bands, tapping along offers them a chance to attend to, appreciate, and even rehearse key aspects of these rappers¡¯ expert performance choices, regardless of how well they might follow by ear.

• Upham, Finn & Christophersen, Bj?rn Morten (2023). Bodies in Concert: RITMO project with the Stavanger symfoniorkester. Full text in Research Archive Show summary

  Discussion between Composer and Music Researcher on the physiological trajectories of orchestra members performing a new and challenging work.

• Upham, Finn (2023). Insight into human respiration through the study of orchestras and audiences. Full text in Research Archive Show summary

  Short presentation to a general science audience about what is uncovered in the coordinated actions of an orchestra by wearable psychophysiology sensors.

• Bishop, Laura & Upham, Finn (2023). Bodies in Concert. doi: https:/www.uio.no/ritmo/english/projects/Bodies-in-Concert/. Full text in Research Archive Show summary

  Increasingly, research on music performance is moving out of controlled laboratory settings and into concert halls, where there are opportunities to explore how performance unfolds in high-arousal conditions and how performers and audiences interact. In this session, we will present findings from a series of live research concerts that we carried out with the Stavanger Symphony Orchestra. The orchestra performed the same program of classical repertoire for four audiences of schoolchildren and an audience of families. Orchestra members wore sensors that collected cardiac activity, respiration, and body motion data, and the conductor additionally wore a full-body motion capture suit and eye-tracking glasses. Audience members in some of the concerts were invited to wear reflective wristbands, and wristband motion was captured using infrared video recording. We will begin the session with a discussion of the scientific and methodological challenges that arose during the project, in particular relating to the large scale of data capture (>50 musicians and hundreds of audience members), the visible nature of research that is carried out on a concert stage, and the development of procedures for aligning data from different recording modalities. Next, we will present findings from two lines of analysis that investigate different aspects of behavioural and physiological coordination within the orchestra. One analysis investigates the effects of audience noise and musical roles on coherence in (i) cardiac rate and variability and (ii) respiratory phase and rate. The second analysis investigates the effects of musical demands on synchronization of body sway, bowing, and respiration in string sections. We will conclude the session with an open discussion of how live concert research might be optimized.

• Martin, Remy Richard; Cross, Ian; Upham, Finn; Bishop, Laura; S?rb?, Solveig & ?land, Frederik (2023). What can one learn from more naturalistic concert research? Full text in Research Archive
• Riaz, Maham; Upham, Finn; Burnim, Kayla; Bishop, Laura & Jensenius, Alexander Refsum (2023). Comparing inertial motion sensors for capturing human micromotion. Full text in Research Archive Show summary

  The paper presents a study of the noise level of accelerometer data from a mobile phone compared to three commercially available IMU-based devices (AX3, Equivital, and Movesense) and a marker-based infrared motion capture system (Qualisys). The sensors are compared in static positions and for measuring human micromotion, with larger motion sequences as reference. The measurements show that all but one of the IMU-based devices capture motion with an accuracy and precision that is far below human micromotion. However, their data and representations differ, so care should be taken when comparing data between devices.

• Lartillot, Olivier; Swarbrick, Dana; Upham, Finn & Cancino-Chac¨®n, Carlos Eduardo (2023). Video visualization of a string quartet performance of a Bach Fugue: Design and subjective evaluation. Full text in Research Archive
• Upham, Finn; Memis, Ahmet Emin; Hansen, Niels Chr.; Rosas, Fernando E.; Clim, Maria-Alena & Jensenius, Alexander Refsum (2022). Participatory applause: Interactions of audience members clapping at the end of a classical music concert. Full text in Research Archive Show summary

  Participatory applause: Interactions of audience members clapping at the end of a classical music concert According to musicological studies of audience culture, applause is the most overt form of participation allowed to the collections of individuals attending classical music concerts (Brandl-Risi, 2011; Small, 1998; Tr?ndle, 2020). The final round of applause can exhibit many interesting dynamics related to their collective enthusiasm for the performance (Lupyan & Rifkin, 2003), the local applause culture, and what is on stage during the clapping. Quantitative empirical study of group clapping behaviours has principally depended on participants clapping on request in laboratory settings or A/V recordings from concerts (Neda, 2000) and presentations (Mann et al., 2013). To study the coordination involved in this collective behaviour, we need accurate measurements of individuals clapping voluntarily in a real concert setting. To describe how the appreciative audience members adjust their clapping to each other and the action on stage during the final round of applause, demonstrating their participation at a concert¡¯s end. * After the Danish String Quartet (DSQ) performed their last piece at the Music Lab Copenhagen Concert, the audience clapped continuously for nearly two minutes. During that time, the musicians stood and bowed, had scientific instruments removed from their bodies, left the stage, returned to bow again, and finally left the stage for good. The clapping action of individual participants in this concert experiment was captured by a mobile phone on their chests, and these recordings show how individuals¡¯ clapping contributed to the collective effect shared with the musicians. Through the final applause interval, 70 devices captured clear clap sequences, representing over half of the audience at this chamber performance. In some ways, their applause followed expected patterns for a concert audience. They began to applaud over a very short time interval (Mann et al., 2013), more than half starting within less than a second of each other. After 20 s of independent clapping at rates from under 120 BMP to over 200 BPM, the participants shifted to clapping together on a shared beat, a practice that is common for Danish audiences. This group maintained synchrony for over a minute while steadily accelerating from around 158 BMP to 176 BMP, an expected consequence of mutual adaptation during group clapping (Thomson et al., 2018). The coordinated action was strongest while the musicians were on the stage but a subset of independent clapping broke out while the audience waited for the performers to return for their final round of bows. Participants¡¯ claps were evaluated from two perspectives: the alignment of claps, reflecting the dominant shift from independence to coordination, and the distribution of participants¡¯ clapping rates over time. Despite some measurement challenges, the shift from independent to coordinate clapping emerges strongly from participants¡¯ movements, with the median rate of clapping slowing until a dominant beat takes hold. Individuals¡¯ clap sequences confirm that the independent clapping at the start of the applause is a result of individual participants clapping isochronously at their own rate, separate in rate and phase from their neighbours in the hall. When the audience claps together, they are voluntarily adjusting to the dominant rate and phase of the people in the hall, with little change in the quality of their isochronous clapping action. Drift in the synchronised clapping rate reflects mutual attentiveness while variation in the number of participants contributing to the coordinated claps suggests differences in applause strategy. Many participants opted to coordinate with their peers while some seemed to prioritise reacting to the musicians.

• Upham, Finn (2022). Audience reconstructed: Fan interactions on twitter during livestreamed BTS concerts. Full text in Research Archive
• Upham, Finn (2022). Respiratory phase alignment in music audiences. Full text in Research Archive
• Upham, Finn; Park, So Yeon & Lee, Jin Ha (2022). Concert tweeting: Textures of fan activity on twitter during Sowoozoo. Full text in Research Archive Show summary

  Live-streamed BTS concerts have become a familiar fan activity for ARMY. While the broadcasting platform gives fans reliable access to the show, these remote audiences are also using alternative platforms to connect with each other. We look at Twitter activity using the #SOWOOZOO tag around the muster concerts in 2021 to understand how and when ARMY twitter interacted during the performance. Liking and retweeting activity followed the attentional demands of the concert live stream. We observed greater fan engagement during the less active times of Sowoozoo (e.g., when BTS were off stage or talking for long periods). This suggested fans did not want to look away from the exciting musical performances but did read and engage with circulating tweets when the program allowed, like in-person audience members checking in with their friends between sets. However, most original posts to the hashtag had a different pattern, reflecting fans¡¯ needs to shout about what was on stage regardless of whether anyone was looking to read their reactions in real time. Although they contained highly relatable expressions of the tweeter¡¯s own strong feelings of love and excitement, these reaction posts received relatively little engagement, buried quickly in the continued heavy flow on ARMY twitter¡¯s timelines. Instead, the most popular tweets to share and like had high quality information like video clips and translations.Tweets in the middle range of engagement often included both expressions of excitement and valuable media. Disinterested in the limited interaction opportunities within the concert streaming platform, these fans adapted Twitter to serve multiple objectives, from screaming their feelings in a virtual crowd and to collecting and sharing the concert-related content they loved.

• Upham, Finn & Oddekalv, Kjell Andreas (2022). Tapping into the flow: Proprioceptive interaction with rap in Rhythm Hive. Full text in Research Archive Show summary

  Rhythm Hive is a mobile rhythm game released that features music by the Hybe bands BTS, Enhyphen, and TXT. With examples of gameplay, we describe how this app gives fans access to new levels of appreciation for rap verses, particularly for non-korean speakers and non-rappers. Interactive proprioception, a concept by game designer Matt Bloch (Miller, 2017), refers to changes in a player's perception of music as they learn movements to it. Choreography draws attention to coincidental elements of a track, allowing players to intimately and viscerally act with a recorded performance. Players¡¯ actions in Rhythm Hive principally parallel the voiced tracks, the members¡¯ contributions, through dense combinations of finger taps, holds, and slides. Taps to sung phrases often follow melodic pitch but the gestures for rap verses must negotiate different criteria for choosing which syllables to tap to and what shape to give mostly-monotone 16th note runs. In a rap flow - here defined as the rhythm of the words and rhymes in a piece of rap music - the rhythmic structure is created by the placement of the rapped syllables, the stress patterns of said syllables, rhymes and phrase boundaries within a music-rhythmic context. Flow, together with delivery (the way the rhythms are performed) and lyrical content constitute the rap¡¯s expression. In Rhythm Hive, players kinesthetically experience structural features of rap flows: how rhythmic patterns are repeated with slight but significant differences, how long phrases cross music-metrical boundaries to build up tension towards an eventual release, and how phrase-endings with rhymes and punchlines are significant gestures. Comparing Easy and Hard mode sequences to select excerpts of the BTS discography, we show how gameplay illuminates the artistry in these verses by tapping along with and therefore into the flow.

• Upham, Finn (2022). Uncovering the active listener. Full text in Research Archive Show summary

  Our experiences of music are both highly idiosyncratic, special to each of us in each moment, and collective, with common influences across a listening crowd. Through a series of empirical studies on how people feel and behave during music listening, Finn Upham traces their trajectory from a basic model of performed stimulus and audience response to an empowered-listener view of musical engagement. If participation is part of all musical experiences, this poses a question to all producers of music: ¡°What are you asking your audience to do?¡±

• Swarbrick, Dana; Upham, Finn; Erdem, Cagri; Jensenius, Alexander Refsum & Vuoskoski, Jonna Katariina (2022). Measuring Virtual Audiences with The MusicLab App: Proof of Concept. Full text in Research Archive Show summary

  We present a proof of concept by using the mobile application MusicLab to measure motion during a livestreamed concert and examining its relation to musical features. With the MusicLab App, participants¡¯ own smartphones¡¯ inertial measurement unit (IMU) sensors can be leveraged to record their motion and their subjective experiences collected through survey responses. The MusicLab Lock-down Rave was an Algorave (live-coded dance music) livestreamed concert featuring prolific performers Renick Bell and Khoparzi. They livestreamed for an international audience who wore their smartphones with the MusicLab App while they listened/danced to the performances. From their acceleration, we computed quantity of motion and compared it to musical features that have previously been associated with music-related motion, namely pulse clarity and low and high spectral flux. By encountering challenges and implementing improvements, the MusicLab App has become a useful tool for researching music-related motion.

• Swarbrick, Dana; Upham, Finn; McAdams, Stephen; Trainor, Laurel & Merrill, Julia (2022). Concert Experiment Research - Reflections on Past and Future. Full text in Research Archive Show summary

  In the twenty years since the seminal concert experiments conducted by Stephen McAdams, many music cognition labs have invested resources and expertise into studying music in a concert setting. Such projects often involve collaborations between teams of researchers and artists, new technologies, challenging measurement conditions, and creative analysis strategies. As music science events, they can be very attractive to the popular press and yet difficult to report in academic circles. Complications from their bid for greater ecological validity can look like a weakness by laboratory standards, but such challenges to common practices and theories may also be a strength of this distinct research paradigm. In this moderated panel, we gather researchers with a range of experiences in researching musical concerts to share lessons learned and hopeful directions for this experimental paradigm. What should we expect to capture from musicians and audiences in these conditions? What research questions can and should be investigated with live performance and joint spectatorship? Can today¡¯s technologies improve the methods used to conduct these studies? The symposium will include short presentations on past research with a focus on methods and strategy, some discussion of the main questions between the panelists, and a substantial portion of time devoted to discussion with the audience, as experience with concert studies is wider than the published record suggests. Specifically, the panel will consist of three sections: 1) introductions of the panelists¡¯ expertise on concert experiment research, 2) question and answer period between panelists, and 3) an audience question period. The panel will consist of four leading researchers in the field of concert studies and it will be hosted by a doctoral researcher with expertise in the field.

• Swarbrick, Dana; Upham, Finn; Erdem, Cagri; Burnim, Kayla & Jensenius, Alexander Refsum (2021). The MusicLab App ¨C Exploring the usage of mobile accelerometry to measure audience movement and respiration. Full text in Research Archive
• Swarbrick, Dana; Upham, Finn; Erdem, Cagri; Burnim, Kayla & Jensenius, Alexander Refsum (2021). MusicLab Algorave ¨C An exploratory study examining the usage of mobile accelerometry to measure movements of a virtual concert audience. Full text in Research Archive
• Upham, Finn; Zelechowska, Agata; Gonzalez, Victor & Jensenius, Alexander Refsum (2021). Quiet Breathing to Heard Music. Full text in Research Archive
• Upham, Finn (2024). Peer Review Report For: Is Taylor Swift leading a new Pop revolution? A cross-generation analysis of Pop/Rock cover songs [version 3; peer review: 1 approved with reservations, 3 not approved]. Taylor & Francis Group. Full text in Research Archive Show summary

  Open review of an study on patterns in cover songs reported in the website SecondHandSongs. Review covers he contemporary cultural context of cover songs and the structure of this database, its linguistic, geographic, legal, and technical biases in this database, besides making specific recommendations for the paper.

View all works in NVA