Alexander Szorkovszky

• Orioma, Charles; Krivan, Josef Jan; Mathema, Rujeena; Silva, Pedro Rego Lencastre e; Lind, Pedro & Szorkovszky, Alexander [Show all 7 contributors for this article] (2025). Identification of fixations and saccades in eye-tracking data using adaptive threshold-based method. arXiv. doi: 10.48550/ARXIV.2512.23926. Full text in Research Archive Show summary

  Properties of ocular fixations and saccades are highly stochastic during many experimental tasks, and their statistics are often used as proxies for various aspects of cognition. Although distinguishing saccades from fixations is not trivial, experimentalists generally use common ad-hoc thresholds in detection algorithms. This neglects inter-task and inter-individual variability in oculomotor dynamics, and potentially biases the resulting statistics. In this article, we introduce and evaluate an adaptive method based on a Markovian approximation of eye-gaze dynamics, using saccades and fixations as states such that the optimal threshold minimizes state transitions. Applying this to three common threshold-based algorithms (velocity, angular velocity, and dispersion), we evaluate the overall accuracy against a multi-threshold benchmark as well as robustness to noise. We find that a velocity threshold achieves the highest baseline accuracy (90-93\%) across both free-viewing and visual search tasks. However, velocity-based methods degrade rapidly under noise when thresholds remain fixed, with accuracy falling below 20% at high noise levels. Adaptive threshold optimization via K-ratio minimization substantially improves performance under noisy conditions for all algorithms. Adaptive dispersion thresholds demonstrate superior noise robustness, maintaining accuracy above 81% even at extreme noise levels ({\sigma} = 50 px), though a precision-recall trade-off emerges that favors fixation detection at the expense of saccade identification. In addition to demonstrating our parsimonious adaptive thresholding method, these findings provide practical guidance for selecting and tuning classification algorithms based on data quality and analytical priorities.

• Bravo, Pedro Pablo Lucas; Szorkovszky, Alexander; Fasciani, Stefano & Glette, Kyrre (2025). HS-ims: A Platform for Human-Swarm Interactive Music Systems. In Tei, Kenji & Zambonelli, Franco (Ed.), 2025 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798331502157. doi: 10.1109/ACSOS-C66519.2025.00067. Full text in Research Archive Show summary

  This paper introduces HS-ims, an integrated virtual environment designed for developing real-time musical multiagent systems. It addresses the need for technological platforms that emphasize sound applications, flexible prototyping, and comprehensive data collection from these multi-agent systems. HS-ims offers a 3D virtual environment with built-in sound synthesis via binaural audio, an Open Sound Control (OSC) API implementation that is language-agnostic, and a data collection module. The platform operates with an External Controller managing multi-agent logic. We present relevant performance metrics and provide examples highlighting how HS-ims has proven valuable in research and education. This encourages further exploration and establishes it as a useful tool for music performance.

• Wallace, Benedikte; Glette, Kyrre & Szorkovszky, Alexander (2025). How can we make robot dance expressive and responsive? A survey of methods and future directions. Frontiers in Computer Science. 7. doi: 10.3389/fcomp.2025.1575667. Full text in Research Archive Show summary

  The development of robots that can dance like humans presents a complex challenge due to the disparate abilities involved and various aesthetic qualities that need to be achieved. This article reviews recent advances in robotics, artificial intelligence, and human-robot interaction toward enabling various aspects of realistic dance, and examines potential paths toward a fully embodied dancing agent. We begin by outlining the essential abilities required for a robot to perform human-like dance movements and the resulting aesthetic qualities, summarized under the terms expressiveness and responsiveness . Subsequently, we present a review of the current state-of-the-art in dance-related robot technology, highlighting notable achievements, limitations and trade-offs in existing systems. Our analysis covers various approaches, including traditional control systems, machine learning algorithms, and hybrid systems that aim to imbue robots with the capacity for responsive, expressive movement. Finally, we identify and discuss the critical gaps in current research and technology that need to be addressed for the full realization of realistic dancing robots. These include challenges in real-time motion planning, adaptive learning from human dancers, and morphology independence. By mapping out current methods and challenges, we aim to provide insights that may guide future innovations in creating more engaging, responsive, and expressive robotic systems.

• Bravo, Pedro Pablo Lucas; Fasciani, Stefano; Szorkovszky, Alexander & Glette, Kyrre (2025). An Interactive Self-Assembly Swarm Music System in Extended Reality. In Sei?a, Mariana & Wirfs-Brock, Jordan (Ed.), AM '25: Proceedings of the 20th International Audio Mostly Conference. Association for Computing Machinery (ACM). ISSN 9798400720659. p. 255¨C269. doi: 10.1145/3771594.3771620. Full text in Research Archive Show summary

  This paper explores the music-making capabilities of a swarm intelligence type of algorithm known as self-assembly in an interactive context using Extended Reality (XR) technologies. We describe the modifications made to a fully autonomous version of this algorithm, which we proposed in a previous work, allowing us to adapt it for user-interactive music. Moreover, we present the design of an XR system that supports this adaptation, modelled as a human-swarm interactive music system, which is implemented in the Meta Quest 3 headset. An auto-ethnographic study was conducted to discover the affordances of the system in a music improvisation session. The study, supported by empirical measurements collected during the session, enables a comparison between the interactive version and the original autonomous offline version, providing valuable insights into how a user can influence the swarm's behaviour. The results are used to discuss the music performance possibilities and future directions for this type of interactive music system.

• Bhandari, Shailendra; Silva, Pedro Rego Lencastre e; Mathema, Rujeena; Szorkovszky, Alexander; Yazidi, Anis & Lind, Pedro (2025). Modeling eye gaze velocity trajectories using GANs with spectral loss for enhanced fidelity. Scientific Reports. 15(1). doi: 10.1038/s41598-025-05286-5. Full text in Research Archive Show summary

  Accurate modeling of eye gaze dynamics is essential for advancement in human-computer interaction, neurological diagnostics, and cognitive research. Traditional generative models like Markov models often fail to capture the complex temporal dependencies and distributional nuance inherent in eye gaze trajectories data. This study introduces a Generative Adversarial Network (GAN) framework employing Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) generators and discriminators to generate high-fidelity synthetic eye gaze velocity trajectories. We conducted a comprehensive evaluation of four GAN architectures: CNN-CNN, LSTM-CNN, CNN-LSTM, and LSTM-LSTM¨Ctrained under two conditions: using only adversarial loss ($$L_G$$) and using a weighted combination of adversarial and spectral losses. Our findings reveal that the LSTM-CNN architecture trained with this new loss function exhibits the closest alignment to the real data distribution, effectively capturing both the distribution tails and the intricate temporal dependencies. The inclusion of spectral regularization significantly enhances the GANs¡¯ ability to replicate the spectral characteristics of eye gaze movements, leading to a more stable learning process and improved data fidelity. Comparative analysis with a Hidden Markov Model (HMM) optimized to four hidden states further highlights the advantages of the LSTM-CNN GAN. Statistical metrics show that the HMM-generated data significantly diverges from the real data in terms of mean, standard deviation, skewness, and kurtosis. In contrast, the LSTM-CNN model closely matches the real data across these statistics, affirming its capacity to model the complexity of eye gaze dynamics effectively. These results position the spectrally regularized LSTM-CNN GAN as a robust tool for generating synthetic eye gaze velocity data with high fidelity. Its ability to accurately replicate both the distributional and temporal properties of real data holds significant potential for applications in simulation environments, training systems, and the development of advanced eye-tracking technologies, ultimately contributing to more naturalistic and responsive human-computer interactions.

• Bravo, Pedro Pablo Lucas; Szorkovszky, Alexander; Fasciani, Stefano & Glette, Kyrre (2024). Self-Assembly and Synchronization: Crafting Music with Multi-Agent Embodied Oscillators. In IEEE, IEEE (Eds.), 2024 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350363876. p. 145¨C150. doi: 10.1109/ACSOS61780.2024.00034. Full text in Research Archive Show summary

  This paper proposes a self-assembly algorithm that generates rhythmic music. It uses multiple pulsed oscillators embedded in cube-shaped agents in a virtual 3D space. When these units connect with each other, their oscillators synchronize, triggering regular sound events that produce musical notes whose sound dynamics change based on the size of the structures formed. This study examines the synchronization time of these oscillators and the emergent properties of the structures formed during the algorithm¡¯s execution. Moreover, the resulting sound, determined by multiple interactions among agents, is analyzed in the time and frequency domains from its signal. The results show that the synchronization time slightly increases when more agents participate, although with high variability. Also, a quasi-regular pattern of increase and decrease in the number of structures over time is observed. Additionally, the signal analysis illustrates the effect of the self-assembly strategy in terms of rhythmical patterns and sound energy over time. We discuss these results and the potential applications of this multi-agent approach in the sound and music field.

• Bravo, Pedro Pablo Lucas; Fasciani, Stefano; Szorkovszky, Alexander & Glette, Kyrre (2024). Interactive Sonification of 3D Swarmalators, Proceedings of the International Conference on New Interfaces for Musical Expression. NIME. p. 252¨C260. doi: 10.5281/zenodo.13904846. Full text in Research Archive Show summary

  This paper explores the sound and music possibilities obtained from the sonification of a swarm of coupled oscillators moving in a virtual space called ¡°Swarmalators¡±. We describe the design and implementation of a Human-Swarm Interactive Music System based on the 3D version of the Swarmalator model, which is used for signal analysis of the overall sound output in terms of scalability; that is, the effect of varying the number of agents in a swarm system. We also study the behaviour of autonomous swarmalators in the presence of one user-controlled agent, which we call the interactive swarmalator. We observed that sound frequencies barely deviate from their initial values when there are few agents, but they diverge significantly in a highly dense swarm. Additionally, with the inclusion of the interactive swarmalator, the group¡¯s behaviour tends to adjust towards it. We use these results to explore the potential of swarmalators in music performance under various scenarios. Finally, we discuss opportunities and challenges to use the Swarmalator model for sound and music systems.

• Veenstra, Frank; Szorkovszky, Alexander & Glette, Kyrre (2023). Decentralized Control and Morphological Evolution of 2D Virtual Creatures. In Iizuki, Hiroyuki; Suzuki, Keisuke; Uno, Ryoko; Damiano, Luisa; Spychala, Nadine; Aguilera, Miguel; Izquierdo, Eduardo; Suzuki, Reiji & Baltieri, Manuel (Ed.), ALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artificial Life Conference. MIT Press. doi: 10.1162/isal_a_00656. Full text in Research Archive
• Corral-Lopez, Alberto; Bloch, Natasha I.; Bijl, Wouter van der; Cortazar-Chinarro, Maria; Szorkovszky, Alexander & Kotrschal, Alexander [Show all 11 contributors for this article] (2023). Functional convergence of genomic and transcriptomic architecture underlies schooling behaviour in a live-bearing fish. Nature Ecology and Evolution. doi: 10.1038/s41559-023-02249-9. Full text in Research Archive Show summary

  Abstract The organization and coordination of fish schools provide a valuable model to investigate the genetic architecture of affiliative behaviours and dissect the mechanisms underlying social behaviours and personalities. Here we used replicate guppy selection lines that vary in schooling propensity and combine quantitative genetics with genomic and transcriptomic analyses to investigate the genetic basis of sociability phenotypes. We show that consistent with findings in collective motion patterns, experimental evolution of schooling propensity increased the sociability of female, but not male, guppies when swimming with unfamiliar conspecifics. This finding highlights a relevant link between coordinated motion and sociability for species forming fission¨Cfusion societies in which both group size and the type of social interactions are dynamic across space and time. We further show that alignment and attraction, the two major traits forming the sociability personality axis in this species, showed heritability estimates at the upper end of the range previously described for social behaviours, with important variation across sexes. The results from both Pool-seq and RNA-seq data indicated that genes involved in neuron migration and synaptic function were instrumental in the evolution of sociability, highlighting a crucial role of glutamatergic synaptic function and calcium-dependent signalling processes in the evolution of schooling.

• Corral-Lopez, Alberto; Kotrschal, Alexander; Szorkovszky, Alexander; Garate-Olaizola, Maddi; Herbert-Read, James & Bijl, Wouter van der [Show all 13 contributors for this article] (2023). Evolution of schooling drives changes in neuroanatomy and motion characteristics across predation contexts in guppies. Nature Communications. 14(1). doi: 10.1038/s41467-023-41635-6. Full text in Research Archive Show summary

  Abstract One of the most spectacular displays of social behavior is the synchronized movements that many animal groups perform to travel, forage and escape from predators. However, elucidating the neural mechanisms underlying the evolution of collective behaviors, as well as their fitness effects, remains challenging. Here, we study collective motion patterns with and without predation threat and predator inspection behavior in guppies experimentally selected for divergence in polarization, an important ecological driver of coordinated movement in fish. We find that groups from artificially selected lines remain more polarized than control groups in the presence of a threat. Neuroanatomical measurements of polarization-selected individuals indicate changes in brain regions previously suggested to be important regulators of perception, fear and attention, and motor response. Additional visual acuity and temporal resolution tests performed in polarization-selected and control individuals indicate that observed differences in predator inspection and schooling behavior should not be attributable to changes in visual perception, but rather are more likely the result of the more efficient relay of sensory input in the brain of polarization-selected fish. Our findings highlight that brain morphology may play a fundamental role in the evolution of coordinated movement and anti-predator behavior.

• Szorkovszky, Alexander; Veenstra, Frank & Glette, Kyrre (2023). Toward cultures of rhythm in legged robots. In Iizuki, Hiroyuki; Suzuki, Keisuke; Uno, Ryoko; Damiano, Luisa; Spychala, Nadine; Aguilera, Miguel; Izquierdo, Eduardo; Suzuki, Reiji & Baltieri, Manuel (Ed.), ALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artificial Life Conference. MIT Press. doi: 10.1162/isal_a_00673. Full text in Research Archive Show summary

  It is widely thought that sensorimotor synchronization, underpinning cultural domains such as music and dance, played a critical role in the evolution of human sociality. Here, we present virtual legged robots controlled by central pattern generators (CPGs) that evolve to synchronize motion to rhythmic sensory input in real time. Multi-stage, multi-objective evolutionary algorithms were used to maximize flexibility of the CPGs with respect to control parameters, and then to optimize a neural input layer for wide-ranging susceptibility to rhythmic inputs. The evolved CPGs self-organize to accommodate the input sequence over a range of frequencies and patterns while keeping the agents upright. We show how this behaviour can be scaled up to multiple interacting agents, including with differing morphologies, to produce novel behaviours. We then outline how spike timing dependent plasticity can be used for the acquisition of new motor patterns. Finally, taking inspiration from biocultural evolution and cognitive neuroscience, we suggest ways in which real-time social adaptation can play a key role in the evolution of complex social behaviours in robots.

• Szorkovszky, Alexander; Veenstra, Frank & Glette, Kyrre (2023). From real-time adaptation to social learning in robot ecosystems. Frontiers in Robotics and AI. 10. doi: 10.3389/frobt.2023.1232708. Full text in Research Archive Show summary

  While evolutionary robotics can create novel morphologies and controllers that are well-adapted to their environments, learning is still the most efficient way to adapt to changes that occur on shorter time scales. Learning proposals for evolving robots to date have focused on new individuals either learning a controller from scratch, or building on the experience of direct ancestors and/or robots with similar configurations. Here we propose and demonstrate a novel means for social learning of gait patterns, based on sensorimotor synchronization. Using movement patterns of other robots as input can drive nonlinear decentralized controllers such as CPGs into new limit cycles, hence encouraging diversity of movement patterns. Stable autonomous controllers can then be locked in, which we demonstrate using a quasi-Hebbian feedback scheme. We propose that in an ecosystem of robots evolving in a heterogeneous environment, such a scheme may allow for the emergence of generalist task-solvers from a population of specialists.

• Szorkovszky, Alexander; Veenstra, Frank & Glette, Kyrre (2023). Central pattern generators evolved for real-time adaptation to rhythmic stimuli. Bioinspiration & Biomimetics. ISSN 1748-3182. 18(4). doi: 10.1088/1748-3190/ace017. Full text in Research Archive Show summary

  Abstract For a robot to be both autonomous and collaborative requires the ability to adapt its movement to a variety of external stimuli, whether these come from humans or other robots. Typically, legged robots have oscillation periods explicitly defined as a control parameter, limiting the adaptability of walking gaits. Here we demonstrate a virtual quadruped robot employing a bio-inspired central pattern generator (CPG) that can spontaneously synchronize its movement to a range of rhythmic stimuli. Multi-objective evolutionary algorithms were used to optimize the variation of movement speed and direction as a function of the brain stem drive and the centre of mass control respectively. This was followed by optimization of an additional layer of neurons that filters fluctuating inputs. As a result, a range of CPGs were able to adjust their gait pattern and/or frequency to match the input period. We show how this can be used to facilitate coordinated movement despite differences in morphology, as well as to learn new movement patterns.

• Gyllingberg, Linn¨¦a; Szorkovszky, Alexander & Sumpter, David (2023). Using neuronal models to capture burst-and-glide motion and leadership in fish. Journal of the Royal Society Interface. ISSN 1742-5689. 20(204), p. 1¨C13. doi: 10.1098/rsif.2023.0212. Full text in Research Archive
• Szorkovszky, Alexander; Veenstra, Frank; Lartillot, Olivier Serge Gabriel; Jensenius, Alexander Refsum & Glette, Kyrre (2023). Embodied Tempo Tracking with a Virtual Quadruped, Proceedings of the Sound and Music Computing Conference 2023. SMC Network . ISSN 9789152773727. doi: 10.5281/zenodo.10060970. Full text in Research Archive Show summary

  Dynamic attending theory posits that we entrain to time-structured events in a similar way to synchronizing oscillators. Hence, a tempo tracker based on oscillators may replicate humans' ability to rapidly and robustly identify musical tempi. We demonstrate this idea using virtual quadrupeds, whose gaits are controlled by oscillatory neural circuits known as central pattern generators (CPGs). The quadruped CPGs were first optimized for flexible gait frequency and direction, and then an additional recurrent layer was optimized for entrainment to isochronous pulses. Using excerpts of musical pieces, we find that the motion of these agents can rapidly entrain to simple rhythms. Performance was found to be partially predicted by pulse entropy, a measure of the sample's rhythmic complexity. Notably, in addition to having wide tempo ranges, the best performing agents can also entrain to rhythms that are periodic but not quantized on a grid. Our approach offers an embodied alternative to other dynamical systems-based approaches to entrainment, such as gradient-frequency arrays. Such agents could find use as participants in virtual musicking environments, or as real-world musical robots.

• Szorkovszky, Alexander; Veenstra, Frank & Glette, Kyrre (2022). Rapid rhythmic entrainment in bio-inspired central pattern generators, 2022 International Joint Conference on Neural Networks (IJCNN). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9781728186719. doi: 10.1109/IJCNN55064.2022.9891909. Full text in Research Archive Show summary

  Entrainment of movement to a periodic stimulus is a characteristic intelligent behaviour in humans and an important goal for adaptive robotics. We demonstrate a quadruped central pattern generator (CPG), consisting of modified Matsuoka neurons, that spontaneously adjusts its period of oscillation to that of a periodic input signal. This is done by simple forcing, with the aid of a filtering network as well as a neural model with tonic input-dependent oscillation period. We first use the NSGA3 algorithm to evolve the CPG parameters, using separate fitness functions for period tunability, limb homogeneity and gait stability. Four CPGs, maximizing different weighted averages of the fitness functions, are then selected from the Pareto front and each is used as a basis for optimizing a filter network. Different numbers of neurons are tested for each filter network. We find that period tunability in particular facilitates robust entrainment, that bounding gaits entrain more easily than walking gaits, and that more neurons in the filter network are beneficial for pre-processing input signals. The system that we present can be used in conjunction with sensory feedback to allow low-level adaptive and robust behaviour in walking robots.

• Levens, Watson; Szorkovszky, Alexander & Sumpter, David (2022). Friend of a friend models of network growth. Royal Society Open Science. 9(10), p. 1¨C17. doi: 10.1098/rsos.221200. Full text in Research Archive

View all works in NVA

• Szorkovszky, Alexander; Veenstra, Frank & Glette, Kyrre (2022). From real-time adaptation to social learning in robots. Full text in Research Archive

View all works in NVA