![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-001.jpg?height=786&width=853&top_left_y=475&top_left_x=608)

# ALIFE 25 

Ciphers of Life

# Companion Proceedings of the Artificial Life Conference 2025 

Edited by
Olaf Witkowski, Alyssa M. Adams, Lana Sinapayen, Manuel Baltieri, Mahdi Khosravy
Summaries ..... 1
1 Towards the Construction of a Chemical Computer Amahury J. López-Díaz, Juan Perez-Mercader, and Carlos Gershenson
5 The institution bootstrapping problem and the psychological roots of institutions Stavros Anagnou, Christoph Salge, and Peter R. Lewis
9 Mapping the Road to Synthetic Ecosystems Victor Maull1, Ricard Solé
13 Structural Cellular Hash Chemistry Hiroki Sayama
17 The Evolutionary Advantage of Guilt
Theodor Cimpeanu, Luís Moniz Pereira, and The Anh Han
21 Transmission Depends on Trajectory: Movement Rules Reshape Spreading Dynamics Michael Chimento and Damien R. Farine
25 Formation of Stable, Scalable Patterns of Heterogeneous Boid Through Elimination of Metastable States Mari Nakamura
29 Collectives from Difference, and Difference from Collectives
Mohsen Raoufi, Heiko Hamann, and Pawel Romanczuk
Late Breaking Abstracts ..... 33
33 Environmentally Driven Selection in Autocatalytic Micro-compartments Bingyuan Lu, Abhishek Sharma, Kristine Laws, and Leroy Cronin
37 BUNCH: A Hierarchical Filtering Algorithm for Identifying Lifelike Entities in Interactive Particle Systems Mario Martinez-Saito
39 Moral Judgment Scaling in Large Language Models Kazuhiro Takemoto
41 Data-Driven Structural Comparison of Behavioral Dynamics in Living and Non-Living Motility Riku Adachi, Hiroki Kojima, and Takashi Ikegami
45 Bridging ALife, Consciousness, and Language Model Auditing with Integrated Information Yosuke Miyanishi, Keita Mitani
49 Hummingbird Kernel: Chaotic Lotka-Volterra Sampling for Bayesian Optimization in High-Dimensional De- ceptive Landscapes
Takahito Murakami, Maya Grace Torii, and Yoichi Ochiair
53 Spreading Online Social Support in an Avatar Communication App Masanori Takano, and Kenji Yokotani
57 A Predictive-Coding-Inspired Variational RNN Model of Focus-Mind-Wandering Transitions under the Free Energy Principle
Henrique Oyama and Jun Tani
61 Investigating dynamics of Neural Cellular Automata in complex systems Vaibhav Mahajan, and Soumya Banerjee
65 LLM-Driven Social Behaviors of Virtual Doubles in Shared Mixed Reality Chris Elvis Leisi, Florian Christoph Bruggisser and Oliver Sahli
69 Population of Thoughts: Exploring Open-Ended Evolution and Discovery for Test-time Compute and Reasoning Bryan Lim, and Adam Gaier
73 Neural Cellular Automata: From Cells to Pixels Ehsan Pajouheshgar, Yitao Xu, Ali Abbasi, Alexander Mordvintsev, Wenzel Jakob, Sabine Süsstrunk
77 Petri Dish NCA Ivy Zhang, Sebastian Risi, and Luke Darlow
81 Embodied Classification with Neural Cellular Automata Iliya Zhechev

83 Rosetta: Real-Time Bio-Hybrid Interface for Protist Collectives Charles Fosseprez
87 Action and Perception in Collective Motion: The Role of Active Sensing and Minimization of Prediction Error Nicolas Bessone, Heiko Hamann, and Andreagiovanni Reina
91 Molecular Assembly Indices to study Natural Metabolic Evolution Jeppe Van den Langenbergh, Carlos Vigil-Vásquez, Bram Spanoghe, Bernard De Baets, and Michiel Stock
93 Quality-Diversity Search for Open-Ended Machine Learning Michiel Stock, Daan Van Hauwermeiren, Bernard De Baets, and Steff Taelman
95 Architecture Design in Boolean Reservoir Computing Christopher Michael Skeide Vibe1, Mikkel E. Lepperødd, Solve Sæbo, and Stefano Nichele
99 Language Cellular Automata: Guiding Morphogenesis with Natural Language Elias Najarro, Eleni Nisioti, Milton L. Montero, Sebastian Risi
103 Visualizing the Structure of Lenia Parameter Space Barbora Hudcová, František Dušek, Marco Tuccio, and Clément Hongler
107 Evolvable Chemotons: Toward the Integration of Autonomy and Evolution Kuzuya Horibe, Daichi G. Suzuki
111 Integrative Formulation of Autonomous and Evolving Aspects of Life by Category Theory Ryuzo Hirota and Daichi G. Suzuki
115 Evolving Interaction Protocols for Open-ended Collective Innovation Eleni Nisioti, Akhi Mocherla, Jérémy Perez, Clément Moulin-Frier, Pierre-Yves Oudeyer, Maxime Derex, Cédric Colas
119 Attack of the Clones: Importance of Diversity in Genetic Programming Hyper-Heuristic Marko Đurasević
123 Why do Trees Grow? An Investigation of Bloat in Genetic Programming Hyper-Heuristics Marko Đurasević
127 Swarm2Algo: Towards Distilling Emergent Collective Behaviors as Interpretable Distributed Algorithms Devendra Parkar1, Raylene A. Faerber1, Hillary Li1, Ping-Yu Duan1, and Joshua J. Daymude
131 Pheromone-mediated movement establishes spatial segregation in simulated ants Maisie Sadler and Seth Bullock
135 Evidence for Open-Ended Evolution in a Binary Cellular Automaton Arend Hintze and Clifford Bohm
139 Spark: Modular Spiking Neural Networks Mario Franco and Carlos Gershenson
143 Learning with Restricted Boolean Models Mario Franco and Carlos Gershenson
147 A Challenging Visual Language Task for the Iterated Learning Model Hyoyeon Lee, Seth Bullock, and Conor Houghton
151 Exploring the Landscape of Non-Equilibrium Memories with Neural Cellular Automata Ehsan Pajouheshgar, Aditya Bhardwaj, Nathaniel Selub, and Ethan Lake
155 Costly information and its role in reducing threshold uncertainty in collective risk dilemmas Genki Ichinose, Mizuki Yokoyama, Wakaba Tateishi, Hiromu Ito, Tom Lenaerts, and Elias Fernández Domingos
159 Computational Evolutionary Psychiatry Account for Evolutionary Paradox in Autism Spectrum Disorder Shumpei Kurosaki, Takafumi Soda, Hiroki Kojima, and Yuichi Yamashita
163 State Transitions Unlock Temporal Memory in Swarm-Based Reservoir Computing Tanner Lund, Alyssa Adams, Nathanael Aubert-Kato, and Takashi Ikegami
167 Motion as the Source of the Spirits: Recording, Preservation, and Generation Aina Kojima, Yuki Tanaka, Kosuke Shikata, Eiko Matsuda, and Seiichiro Katsura
169 Society of Mind on Humanoid ALTER3 with Concurrent Modular Agents Takahide Yoshida, Norihiro Maruyama, Atsushi Masumoti, and Takashi Ikegami
173 H-Lenia: A Hierarchical Extension of Lenia to Analyze Multi-level Emergence through Inter-level Feedback Tatsuki Furukawa, Takaya Arita and Reiji Suzuki

177 Modeling Emergent Patterns of Life in Dynamic Galactic Systems Harrison B. Smith and Lana Sinapayen
181 Towards Physicochemical Foundations of Chemical Learning Marco Tuccio, Jason W. Rocks, and Joshua E. Goldford
185 Of Flesh and Code: Alife-meets-Blife Project Juan M. Nadales, Hiroki Kojima, and Liam M. Longo
187 nferring Spatial Dynamics through Interpretable Neural Cellular Automata Jakob H. Schauser, and Ala Trusina
191 Robots and Time: Synthesizing Timescales for Embodied Artificial Systems Suet Lee and Heiko Hamann
195 Neural Particle Automata: Learning Self-Organizing Particle Systems Hyunsoo Kim, Ehsan Pajouheshgar, Sabine Süsstrunk, and Jinah Park
199 Collective Behavior Emerging from Social Learning Strategies and Network Structures Jingyu Xi, Wataru Toyokawa
203 A test-bed for open-ended evolution and skill acquisition for embodied collectives Max Oltmanns, and Jonas Kuckling
207 Oscillatory Dynamics as an Universal Substrate for Computation: From Neural Circuits to Analog Electronics and Beyond Felix Effenberger
211 Quantifying Microslips in Tele-operarion Task Takaya Mori, Yuki Tanaka, Seiichiro Katsura, and Eiko Matsuda
215 Using Human Feedback and Reward Modeling to Search for Artificial Life Eugéne Bergeron and Vassilis Papadopoulos
219 Functional Information Decomposition: A Principled Approach to Information Decomposition Clifford Bohm
223 QD-LEAR: Exploring Quality-Diversity Tradeoffs in LLM-Evolved Agent Rule Representations Can Gurkan, Narasimha K. Jwalapuram, Kevin Wang, Rudy Danda
227 Window Into Another World: Simulating Particle Life in the Fractional Dimension of a Sierpinski Gasket Isaac Gendelman MD
231 Self-Organized Convolutional Networks using Meta-Neural Cellular Automata with Linear Attention Meet Barot
235 Depression in the Network: A Study of Social Interactions in Agent Populations Abi M. Sipes, Catherine M. Bain, Nathaniel Kremer-Herman, and Ankur Gupta
239 Simulated Axon Growth for Wiring Patterned Neural Networks Jamieson Warner
243 Peer Evaluation of Individuals in a Language-Based Evolutionary Algorithm Thomas Gabor
247 Entropy Coupled Trait ODEs (ECTO) Reveal Structured Population Dynamics in Longitudinal Psychometric Data Anderson M. Rodriguez
251 Discovering and Controlling Diverse Self-Organised Patterns in Cellular Automata using Autotelic Reinforcement Learning Marko Cvjetko, Gautier Hamon, Clement Moulin-Frier, Pierre-Yves Oudeyer
255 Developmental Pattern Formation in Neural Reaction-Diffusion Systems Salvador Buse and Erik Winfree
259 Internalized Culture versus External Rules: The Emergence of Bounded Rationality in LLM Agent Societies Ryosuke Takata, Atsushi Masumori, and Takashi Ikegami
263 Error-Driven Control of Ants Learning-Walks Thomas Misiek, Andrew Philippides, Paul Graham, and James Knight
267 Adversarial Environment Evolution for Robust Pathfinding Neural Cellular Automata Sam Earle and Julian Togelius

271 DIRT: The Distributed Intelligent Replicator Toolkit
Chengrui Wang, Chase Van Amburg, Chloe Huangyuan Su, Joseph Bejjani, Yasin Mazloumi, Naeem Khoshnevis, Sham Kakade, Kiante Brantley, Aaron Walsman
275 Developmental Dynamics and Information Flows in Growing Neural Cellular Automata Hiroki Sato, Atsushi Masumori, and Takashi Ikegami
279 Learning navigation on the spatial-social interface Patrick Govoni, and Pawel Romanczuk
283 Self-Organising Digital Circuits
Marcello Barylli1, Gabriel Béna, Alexander Mordvintsev, Eleni Nisioti and Sebastian Risi
287 Scaffolding development via morphogenetic gradients
Milton L. Montero, Elias Najarro, Eleni Nisioti and Sebastian Risi
291 Evolution as an alternative to Reinforcement Learning for Continual Control Eleni Nisioti, Kathrin Korte, Sebastian Risi
295 Swarm Size Estimation using Diffusion Equations Loona Macabre, Anthony J. Genot, Leo Cazenille, Nathanaël Aubert-Kato, and Nicolas Bredeche
299 Learning Global Rules from Local Patches: Scaling Neural Cellular Automata Aleksandr Groznykh
301 Gridarians: An LLM-Driven Artificial Life Platform for Exploring Brain-Body Coevolution Can Gurkan and Uri Wilensky
305 Towards Internalist and Socio-Affective Models of Cultural Evolution Erwan Plantec, Jayd Matyas, Eva Mardus, Yoko Urano and Daan Laméris
309 UIUA: A Modern Array Language for Artificial Life Research Noah Syrkis
313 Investigating the effect of network topology in Continual Reinforcement Learning Kathrin Korte, Eleni Nisioti and Sebastian Risi
317 Strategy Extraction from Reinforcement Learning via LLM-Guided Natural Language Annotation Ilya Horiguchi, Michael Crosscombe, Ryosuke Takata, and Takashi Ikegami
321 Lowering Collective Free Energy for Consensus in LLM Agent Collectives: Common Ground via Collective Predictive Coding
Yusuke Hayashi, Kazuya Horibe, and Kotaro Sakamoto
325 What Makes Quantum Biology Quantum - and Is It Necessary For Life? Moritz Kriegleder, Andrea Loettgers and Tarja Knuuttila
329 Communication Matters: LLM Simulations of Cooperation in El Farol Bar Problem Ryosuke Asada, Keita Nishimoto, Kimitaka Asatani, Ichiro Sakata

# Towards the Construction of a Chemical Computer 

Amahury J. López-Díaz ${ }^{1}$, Juan Perez-Mercader ${ }^{2,3}$, and Carlos Gershenson ${ }^{1}$<br>${ }^{1}$ School of Systems Science and Industrial Engineering, Binghamton University, 4400 Vestal Pkwy E, Binghamton, NY 13902, USA<br>${ }^{2}$ Department of Earth and Planetary Sciences, Harvard University, 20 Oxford St, Cambridge, MA 02138, USA<br>${ }^{3}$ Santa Fe Institute, 1399 Hyde Park Rd, Santa Fe, NM 87501, USA<br>alpez@binghamton.edu


#### Abstract

Recent research shows that a single Belousov-Zhabotinsky (BZ) oscillator can perform universal computation at the linear bounded automaton level, suggesting the feasibility of a non-hybrid, programmable chemical computer. Nevertheless, the computational capabilities of coupled biochemistry-free oscillators have remained largely unexplored. Our study reveals that configurations involving two strongly coupled native non-biochemical automata consistently outperform individual non-biochemical automata in forecasting increasingly complex time series. This indicates significant computational benefits from coupling oscillators, especially for recognizing complex computable languages. We discuss the broader implications of these findings for programmable chemical computing, metabolic biocomputing, and resource-bounded algorithmic networks.


Data/Code: https://shorturl.at/h2ylC

## Introduction

Unconventional computing architectures are increasingly explored to mitigate the high energy demands of Artificial Intelligence systems (Kaack et al., 2022). Among these, chemical computing stands out due to its potential for reducing power consumption (Adamatzky and Costello, 2002), and for parallelism (Lobato-Dauzier et al., 2020). The Belousov-Zhabotinsky (BZ) reaction has emerged as particularly promising, with a single BZ oscillator demonstrating universal computation at the linear-bounded-automaton level (Dueñas-Díez and Pérez-Mercader, 2019).

Although the theoretical computational capacity of a single BZ oscillator is established, the potential practical advantages of coupling multiple biochemistry-free oscillators remain unclear (Liu and Parhi, 2022; Baltussen et al., 2024; Lobato-Dauzier et al., 2024). Motivated by ongoing research into cellular supremacy in biocomputing (Grozinger et al., 2019) and resource-bounded algorithmic networks (Abrahao et al., 2019), this study investigates whether coupled BZ oscillators can surpass in performance individual oscillators in processing complex inputs.

Framing the problem as a minimization task-analogous to input acceptance by a chemical Turing machine (Dueñas-

Díez and Pérez-Mercader, 2019)-allows us to translate experimental chemical computation results into a quantifiable time-series forecasting problem. Forecasting accuracy serves as a direct benchmark for evaluating the computational difficulty encountered by different oscillator configurations, thus bridging experimental insights and computational predictions as shown by Ushio et al. (2023).

## Methods

We use reservoir computing (RC), a machine learning method particularly suitable for dynamical system-based forecasting tasks (Pathak et al., 2018). RC comprises a fixed dynamical system (reservoir) where only the weights of the output layer are trained, requiring minimal computational resources (Figure 1). Instead of a recurrent neural network, we used an extension of the Oregonator model (Field and Noyes, 1974) as a reservoir, which reflects experimentally observed synchronization phenomena in coupled BZ oscillators. This model uses ordinary differential equations and fine-tuned parameters suitable for analyzing coupled and uncoupled oscillator scenarios (Liu et al., 2022).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-006.jpg?height=316&width=563&top_left_y=1779&top_left_x=1215)
Figure 1: Reservoir Computing architecture. The reservoir consists of fixed connections (thin solid arrows) from input to reservoir ( $\mathbf{W}_{\text {in }}$ ) and within the reservoir ( $\mathbf{W}_{r}$ ), while only the output connections ( $\mathbf{W}$, dotted thick arrows) are trained. The reservoir is usually a recurrent neural network, but any dynamical system can be used (Tanaka et al., 2019).

To systematically assess how interaction complexity influences the emergence of collective computation, we fol-
lowed Liu et al. (2022) and evaluated four configurations: single oscillator, two uncoupled oscillators, two weakly coupled oscillators, and two strongly coupled oscillators. Using the approach of Ushio et al. (2023), we evaluated each reservoir by generating synthetic time series with varying complexity from the Lorenz attractor (Figure 2) and comparing it with Ridge regression (non-reservoir method). Forecasting performance was measured using Root Mean Squared Error (RMSE).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-007.jpg?height=485&width=575&top_left_y=687&top_left_x=280)
Figure 2: Four different time series associated with different levels of signal complexity. In this context, by complexity we mean a high degree of density in the oscillations of the time series (Ushio et al., 2023). Such oscillatory levels are denoted by $t$.

## Results

For simpler time series (lower oscillatory levels), our results indicate that a single oscillator (or a non-reservoir method) is sufficient in these scenarios. However, as complexity increased (higher oscillatory levels), configurations with two oscillators consistently outperformed a single oscillator, establishing a clear efficiency hierarchy: strongly coupled > weakly coupled $\approx$ uncoupled $>$ single oscillator. Regardless the conditions, a complexity threshold always emerged, beyond which single oscillators failed to maintain accuracy, whereas two-oscillator systems continued performing effectively (Figure 3).

In deterministic scenarios, two-oscillator configurations became largely invariant to initial conditions beyond the identified complexity threshold, suggesting robust internal synchronization and superior representational capacity. In stochastic scenarios involving various noise types (white, pink, brown, blue, violet), the global efficiency hierarchy persisted (two oscillators > single oscillator > no reservoir), although local hierarchies among two-oscillator configurations became less distinct. This indicates complex dependencies on initial synchronization but robust overall performance. The fact that this global hierarchy persisted across deterministic and stochastic conditions, regardless of the

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-007.jpg?height=488&width=659&top_left_y=278&top_left_x=1158)
Figure 3: Forecasting efficiency (average RMSE) across different reservoir configurations. R: Ridge regression; S: Single oscillator; U: Uncoupled oscillators; CL: Weakly coupled oscillators; CH: Strongly coupled oscillators. Dashed vertical line indicates complexity threshold distinguishing single from coupled oscillators effectiveness. Doubleheaded arrows highlight efficiency differences. Note the close performance among two-oscillator configurations, which prevents the visibility of U and CL curves, which are under the CH curve.

parameters used to train the reservoir, indicates robustness against environmental noise.

## Discussion and Conclusion

Our findings establish that coupling chemical oscillators enhances computational capabilities for complex tasks, with strongly coupled configurations demonstrating higher performance after a threshold. This reveals their potential as foundational architectures for scalable, universally computational, and complex-problem-solving chemical computers, potentially addressing even NP-hard problems by leveraging the bistable, reactive and computational nature of the BZ reaction for building scalable Ising machines (Lucas, 2014).

Additionally, our approach offers insights into metabolic biocomputing, suggesting that oscillator coupling may facilitate adaptive responses in primitive metabolic networks (Cervera et al., 2019). Furthermore, these empirical findings enrich theoretical models of algorithmic networks by demonstrating clear computational benefits of resourcebounded chemical automata networks, extending traditional Turing machine-based frameworks (Abrahao et al., 2019).

Broader implications include developing energy-efficient, non-hybrid chemical computing platforms exploiting multioscillator interactions and exploring conditions under which analog networks surpass single automata. Future work will experimentally validate these computational predictions and further explore generalized oscillator network topologies, advancing towards sustainable, scalable, and programmable chemical computing solutions.

## References

Abrahao, F. S., Wehmuth, K., and Ziviani, A. (2019). Algorithmic networks: Central time to trigger expected emergent openendedness. Theoretical Computer Science, 785:83-116.

Adamatzky, A. and Costello, B. D. L. (2002). Experimental logical gates in a reaction-diffusion medium: The XOR gate and beyond. Physical Review E, 66(4):046112.

Baltussen, M. G., de Jong, T. J., Duez, Q., Robinson, W. E., and Huck, W. T. (2024). Chemical reservoir computation in a selforganizing reaction network. Nature, 631(8021):549-555.

Cervera, J., Manzanares, J. A., Mafe, S., and Levin, M. (2019). Synchronization of bioelectric oscillations in networks of nonexcitable cells: from single-cell to multicellular states. The Journal of Physical Chemistry B, 123(18):3924-3934.

Dueñas-Díez, M. and Pérez-Mercader, J. (2019). How chemistry computes: Language recognition by non-biochemical chemical automata. from finite automata to turing machines. Iscience, 19:514-526.

Field, R. J. and Noyes, R. M. (1974). Oscillations in chemical systems. IV. limit cycle behavior in a model of a real chemical reaction. The Journal of Chemical Physics, 60(5):1877-1884.

Grozinger, L., Amos, M., Gorochowski, T. E., Carbonell, P., Oyarzún, D. A., Stoof, R., Fellermann, H., Zuliani, P., Tas, H., and Goñi-Moreno, A. (2019). Pathways to cellular supremacy in biocomputing. Nature communications, 10(1):5250.

Kaack, L. H., Donti, P. L., Strubell, E., Kamiya, G., Creutzig, F., and Rolnick, D. (2022). Aligning artificial intelligence with climate change mitigation. Nature Climate Change, 12(6):518-527.

Liu, X. and Parhi, K. K. (2022). Reservoir computing using dna oscillators. ACS Synthetic Biology, 11(2):780-787.

Liu, Y., Pérez-Mercader, J., and Kiss, I. Z. (2022). Synchronization of Belousov-Zhabotinsky oscillators with electrochemical coupling in a spontaneous process. Chaos: An Interdisciplinary Journal of Nonlinear Science, 32(9).

Lobato-Dauzier, N., Baccouche, A., Gines, G., Lévi, T., Rondelez, Y., Fujii, T., Kim, S., Aubert-Kato, N., and Genot, A. (2024). Neural coding of temperature with a dna-based spiking chemical neuron. Nature Chemical Engineering, 1(8):510-521.

Lobato-Dauzier, N., Cazenille, L., Fujii, T., Genot, A., and AubertKato, N. (2020). Temperature-based inputs for molecular reservoir computers. In Artificial Life Conference Proceedings 32, pages 420-422. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info....

Lucas, A. (2014). Ising formulations of many NP problems. Frontiers in physics, 2:5.

Pathak, J., Hunt, B., Girvan, M., Lu, Z., and Ott, E. (2018). Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach. Physical review letters, 120(2):024102.

Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., and Hirose, A. (2019). Recent advances in physical reservoir computing: A review. Neural Networks, 115:100-123.

Ushio, M., Watanabe, K., Fukuda, Y., Tokudome, Y., and Nakajima, K. (2023). Computational capability of ecological dynamics. Royal Society Open Science, 10(4):221614.

# The institution bootstrapping problem and the psychological roots of institutions 

Stavros Anagnou ${ }^{1,2}$, Christoph Salge ${ }^{1}$, and Peter R. Lewis ${ }^{2}$<br>${ }^{1}$ Adaptive Systems Research Group, University of Hertfordshire, United Kingdom<br>${ }^{2}$ Trustworthy AI lab, Ontario Tech University, Canada<br>s.anagnou@herts.ac.uk

## The institution bootstrapping problem

Institutions are rule systems that play a critical role in enabling communities to manage common-pool resources (e.g., grazing lands, fisheries, or even Minecraft servers) and help them avoid antisocial outcomes by regulating individual self-interest. Prior research suggests institutions emerge when universal participation yields greater collective benefits than non-cooperation. (Ostrom, 1990; Frey and Sumner, 2019). However, a fundamental issue arises: individuals typically perceive participation as advantageous only after an institution is established, creating a paradox-how can institutions form if no one will join before a critical mass exists? We term this conundrum the institution bootstrapping problem and propose that misperception-specifically, agents' erroneous belief that an institution already ex-ists-could resolve this paradox. By integrating welldocumented psychological phenomena-including cognitive biases and perceptual noise-into a game-theoretic framework, we demonstrate how these factors collectively mitigate the bootstrapping problem (Anagnou et al., 2025). Notably, unbiased perceptual noise (e.g., noise arising from agents' differing heterogeneous physical or social contexts) drastically reduces the critical mass of cooperators required for institutional emergence. These findings from challenge conventional assumptions about rationality in institutional design, highlighting how "noisy" cognition can paradoxically enhance cooperation. Our analysis underscores the importance of incorporating human-like cognitive con-straints-not just idealized rationality-into models of institutional emergence and resilience.

## An evolutionary game-theoretic model of an institution

There are many empirical examples of societies being able to avoid anti-social outcomes by devising institutional rules that govern their interactions in the use of common-pool resources (Ostrom, 1990). Example rules include how much water an individual may take from a shared irrigation system, when they may take it, how often they must perform maintenance, etc. We model actions that obey the rules that
preserve the Common-pool Resource (CPR) as Cooperating (C) and selfish actions that exploit the CPR as Defecting (D) e.g., taking too much water. Successful institutions also involve agents taking actions to make sure other agents follow the rules (Powers et al., 2018). We model this as Cooperator Monitors (CM) that take actions to maintain the resource (as done in C) and go out of their way to punish individuals that undermine the institution. Each strategies' utility function depends on the amounts of other strategies in the population. Crucially, the utility of joining the institution depends on how many monitors exist. The more monitors exist, the more you will be punished for not joining. Looking at 1a we see that there is a critical mass of CM individuals needed for everyone to join an institution (for the system to move toward C/CM). This sets up the paradox: if individuals will only join the institution if it exists (threshold of CM reached) then how do institutions come about in the first place?

## Incorporating perception into evolutionary game theory, an unlikely solution to the bootstrapping problem?

How do we get around the bootstrapping problem? One can assume an influx of cooperators/monitors into the group which then incentivises others to join, or a strong leader type emerges which forces the requisite number of individuals to join (Powers et al., 2018). However, an unexplored avenue in addressing this problem may come from incorporating facets of human psychology into theoretical modelling, which would then allow us to try an unorthodox approach: Can agents merely pretending an institution exists, make it a reality?

Firstly we highlight work showing that bias, even though it may be misrepresentative of reality, may be advantageous to individuals. For example, in a frequency-dependent hawk-dove game, where the prevalence of doves and hawks influences strategic payoffs, overestimating the reward of a hawk action can mislead unbiased individuals into perceiving their own actions as less advantageous. As a result, they alter their behaviour, indirectly benefiting biased hawkstrategy players, who now face fewer competitors (McNa-
mara et al., 2021).
In the above case, even though biased individuals were better off, bias was detrimental for group utility, but there are also cases where it is beneficial for the group as well. Davies et al. (2011) show that incorporating an adaptive bias in how one's utility is perceived in a coordination problem can change the attractor landscape of a system, widening the basin of the global optimum and therefore making it more likely for collective systems to arrive at states that benefit all. We take inspiration from the above to reduce the number of individuals needed to set up an institution.

## Results and Discussion

Evolutionary game theory often assumes a perfect perception of what would often be hidden variables e.g. the amount of CM monitors in the population at any given time, which determine the utility difference between joining (C or CM) or not joining (D) the institution.

This assumption of perfect perception is rarely the case, due to limited cognitive resources and information, agents typically rely on imperfect but frugal heuristics e.g. loss aversion bias (Mcnamara and Houston, 1992; Tversky and Kahneman, 1992).

We incorporate such a bias into our model as a proportional bias on the perception of punishment and see if this lowers the threshold needed to start an institution.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-011.jpg?height=330&width=809&top_left_y=1418&top_left_x=169)
Figure 1: Simplex showing the trajectory of change in terms of strategy for each population state in terms of their fitness. (a) Control (b) Agents overestimate the expected cost of punishment. Note, D is not joining the institution. C is taking part in the institution and CM is also taking part in the institution as well and punishing D.

We see in Fig.1b, where agents overestimate the risk of being punished, that the defector attractor indeed shrinks, making it easier for influxes of cooperators or other extrinsic shocks to the system to be able to meet the critical mass of joiners needed to form an institution.

Another factor we wish to explore is uncertainty. It has been established that individual agents are bounded (Simon, 1955) and often limited to the information around them. Therefore they often anchor on local estimates and recent experiences (Simon, 1955; Pachur et al., 2012). These estimates are subject to noise for a host of different factors e.g.
different life experiences, affective make ups of each individual, social networks, social media bubbles e.t.c. (Sundh, 2024; Kahneman et al., 2021; Pachur et al., 2012). In our model this would lead to some agents overestimating the probability of being punished and some underestimating it, with it cancelling it out to the mean at the group level.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-011.jpg?height=279&width=808&top_left_y=612&top_left_x=1089)
Figure 2: Simplex showing the trajectory of change in terms of strategy for each population state in terms of their fitness. (a) Control (b) noisy perception in range [0.125:8] centred at 1 .

To capture this heterogeneity, we incorporate a noisy bias into each individual our model and show that, despite the noise being unbiased at the group level, it also decreases the threshold needed to join an institution 2b. This is surprising: Why does unbiased noise lead to a biased outcome? In fact, despite the noise's unbiased nature at the group level, there is an asymmetry in the system due to the proportional nature of the bias. To create a bias for each agent we either make them underestimate the expected cost of being punished by taking the product of a value $<1$, or make them overestimate it by taking it to a product of a value $>1$. Underestimating when the number of monitors is near zero does not change the estimate much, but overestimating has a larger net effect. Thus overpowering the effect of underestimation. Further, when we make the bias absolute instead of proportional, the evolutionary pathways completely change; with the shifts in strategy space becoming more uncertain and the barriers between regions blur together (see (Anagnou et al., 2025)).

In conclusion, we show that incorporating limitations of bounded agents and psychological biases can make institutions more likely to form. An interesting implication for the design of multi-agent systems is that having accurate information for agents may not necessarily be beneficial for institution formation. Further, this sets up an approach of studying psychological aspects of being an agent to understand the psychological/biological roots of institutions. Further, since the relation between institutions and agents' psychology is not uni-directional: Future research can use agent based models with evolvable biases/noise Anagnou et al. (2023) in their perception to examine how social structures such as institutions and norms shape agent psychology i.e. the normative roots of psychology/biology.

## Acknowledgements

Stavros Anagnou was supported by a PhD studentship from the University of Hertfordshire and a MITACS Globalink Research Award grant. This research was made possible, in part, thanks to the Canada Research Chairs program. We would also like to thank the anonymous reviewers for their feedback on the manuscript and Elias Fernández Domingos for advice regarding the EGTtools library Fernández Domingos et al. (2023).

## References

Anagnou, S., Polani, D., and Salge, C. (2023). The Effect of Noise on the Emergence of Continuous Norms and its Evolutionary Dynamics. In Proceedings of the ALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artificial Life Conference, page 125, Sapporo, Japan. MIT Press.

Anagnou, S., Salge, C., and Lewis, P. R. (2025). Uncertainty, bias and the institution bootstrapping problem. arXiv:2504.21579 [cs].

Davies, A. P., Watson, R. A., Mills, R., Buckley, C. L., and Noble, J. (2011). "If You Can't Be With the One You Love, Love the One You're With": How Individual Habituation of Agent Interactions Improves Global Utility. Artificial Life, 17(3):167-181.

Fernández Domingos, E., Santos, F. C., and Lenaerts, T. (2023). EGTtools: Evolutionary game dynamics in Python. iScience, 26(4):106419.

Frey, S. and Sumner, R. W. (2019). Emergence of integrated institutions in a large population of self-governing communities. PLOS ONE, 14(7):e0216335.

Kahneman, D., Sibony, O., and Sunstein, C. R. (2021). Noise: a flaw in human judgment. William Collins, London.

Mcnamara, J. and Houston, A. (1992). Risk-sensitive foraging: A review of the theory. Bulletin of Mathematical Biology, 54(2-3):355-378.

McNamara, J. M., Houston, A. I., and Leimar, O. (2021). Learning, exploitation and bias in games. PLOS ONE, 16(2):e0246588.

Ostrom, E. (1990). Governing the commons: the evolution of institutions for collective action. The Political economy of institutions and decisions. Cambridge University Press, Cambridge ; New York.

Pachur, T., Hertwig, R., and Steinmann, F. (2012). How do people judge risks: Availability heuristic, affect heuristic, or both? Journal of Experimental Psychology: Applied, 18(3):314330.

Powers, S. T., Ekárt, A., and Lewis, P. R. (2018). Modelling enduring institutions: The complementarity of evolutionary and agent-based approaches. Cognitive Systems Research, 52:6781.

Simon, H. A. (1955). A Behavioral Model of Rational Choice. The Quarterly Journal of Economics, 69(1):99.

Sundh, J. (2024). Human behavior in the context of low-probability high-impact events. Humanities and Social Sciences Communications, 11(1):902.

Tversky, A. and Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5(4):297-323.

# Mapping the Road to Synthetic Ecosystems 

Victor Maull ${ }^{1}$, Ricard Solé ${ }^{1,2}$,<br>${ }^{1}$ ICREA-Complex Systems Lab, Universitat Pompeu Fabra, Barcelona, Spain<br>${ }^{2}$ Santa Fe Institute, Santa Fe, NM, USA<br>ricard.sole@upf.edu, victor.maull@upf.edu


#### Abstract

Ecosystem science has long used population dynamics to link species interactions with system-level behavior. Synthetic biology now enables the design of engineered ecosystems, from microbial consortia to large-scale habitats, employing genetically modified strains to test ecological principles and address applied challenges. We survey modeling approaches and experiments across three classes and scales: microcosms, closed ecospheres, and large-scale ecosystem interventions. A proposed "synthetic ecosystem morphospace" defined by species diversity, community development, and human intervention provides a framework for research, design, and applications, ranging from basic studies to planetary stewardship.


Ecology's quantitative foundation, rooted in early works by Alfred Lotka [Lotka $(1910,1932)$ ], models how species abundances vary across time and space. These principles, traditionally applied to natural communities, have also been tested in simplified experimental systems such as microcosms: small, replicated ecosystems under controlled conditions.[Beyers and Odum (1993)]. The experimental tractability of these simplified systems, together with the advent of synthetic biology, has enabled a new class of experimental ecosystems: those intentionally designed with genetically modified organisms. These systems allow for precise testing of theoretical predictions. From competition or cooperation and spatial dynamics [Mitri et al. (2016); Amor et al. (2017)] see Fig. 1a, to more complex settings, including the creation of closed ecospheres for life-support systems (Fig. 1b) [Gitelson et al. (2002)], and interventions in natural ecosystems (Fig. 1c)[Cases and de Lorenzo (2005); Jansson et al. (2023)]. Scaling from controlled microcosms to large-scale synthetic biospheres raises both opportunities and challenges. Our work advocates for real discussion on the possibility and necessity of creating such new-tonature ecosystems, that we understand as: existing ecosystems, critically endangered by anthropogenic impacts, that can benefit from genetically modified organisms introduction (i.e. microorganisms) to restore basic functions that are rapidly reducing or lacking and poising them close to catastrophic tipping points of no return. Examples of such are rapidly expanding drylands across the world or the plasti-
sphere in the oceans [Berdugo et al. (2022); Zettler et al. (2013)]. Although low-diversity systems are highly predictable, biosphere-scale interventions involve an explosion in terms of complexity, where nonlinear interactions, alternative stable states, or emergent phenomena limit predictability. However, fundamental constraints, derived from network theory, invasion ecology or species-environment coupling mechanisms, can guide the design of robust and functional synthetic ecosystems [Jones et al. (1994); Maull and Solé (2022, 2024); Maull et al. (2024)].

In Fig. 1, we define a roadmap for synthetic ecosystem design into a three-dimensional morphospace defined by: species diversity, degree of community development, and human intervention level. Morphospaces are qualitative mappings that allow us to explore the space of the possible given the principal constraints that govern a case study [Solé (2024)]. It is often possible to find empty spaces that usually correspond to non-observed or impossible solutions. They can also exhibit sharp boundaries that separate qualitatively different classes of structures. Here, three well-differentiated scales appear as nested cubes. At the lower scale, the inner cube is constituted by ecosystems of low complexity yet highly predictable by mathematical modelling, rendering predictions of system attractors matching experimental evidence (Fig. 1a). In Fig. 1b, a synthetic biology expansion of BIOS-3 closed system [Gitelson et al. (1989)] occupies a high-intervention, mediumdiversity, low-succession region due to its human-managed plant-microbe composition. It stands as an example in the volume for closed ecospheres, pivotal to test multispecies facing environmental constraints, representing the logical step towards higher complexity. Finally, the space for bioengineered ecosystems emerges (Fig. 1c), bounded by the sharp divide from novel ecosystems: human-transformed systems combining native and non-native species [Hobbs et al. (2009)] where synthetic biology and intentional design are absent. This paper reviews ecosystems fully or partially designed with genetic engineering, outlines qualitative design approaches, and identifies priorities for safe and effective ecosystem engineering [Solé et al. (2024)].

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-015.jpg?height=1779&width=1529&top_left_y=296&top_left_x=267)
Figure 1: Synthetic ecosystems morphospace. A space is defined using three axes, namely: the degree of community development, species diversity, and the amount of human intervention. Each sphere represents an example, and locations are just relative to each other. The three scales considered in this paper appear as three nested cubes ( $C_{1} \subset C_{2} \subset C_{3}$ ). Novel and synthetic domains are indicated by the grey and yellow areas, respectively. On the left, we have the test-tube scale designs that involve standard motifs, small-scale microbiomes living in man-made artefacts. In a1) and a2), we can see examples on competition and cooperation [Mitri et al. (2016); Amor et al. (2017)], and mesocosm experiments and life support systems, such as BIOS-3, are displayed in detail in b) [Gitelson et al. (2002)]. These two scales are shown within the larger cube $C_{1}$ (right) that contains undisturbed and designed ecosystems as two corners on the right upper part. Example c) stands as a modeling abstraction for bioengineered drylands [Maull and Solé (2024)]. Grey spheres stand for human-driven ecosystems, and blue marbles indicate scenarios of ecosystem intervention.

## Acknowledgments

We thank Jordi Pla, Daniel R. Amor, and Núria Conde, co-authors of the original paper summarized here. Special thanks to L. Hasenbosch for inspiration. This work has been funded by the Ajuntament de Barcelona and "la Caixa" Foundation.

## References

Amor, D. R., Montañez, R., Duran-Nebreda, S., and Solé, R. (2017). Spatial dynamics of synthetic microbial mutualists and their parasites. PLOS Computational Biology, 13(8):e1005689.

Berdugo, M., Vidiella, B., Solé, R. V., and Maestre, F. T. (2022). Ecological mechanisms underlying aridity thresholds in global drylands. Functional Ecology, 36(1):4-23.

Beyers, R. J. and Odum, H. T. (1993). Ecological Microcosms. Springer Advanced Texts in Life Sciences. Springer, New York, NY, 1993 edition.

Cases, I. and de Lorenzo, V. (2005). Genetically modified organisms for the environment: stories of success and failure and what we have learned from them. International Microbiology, 8:213-222.

Gitelson, I., Terskov, I., Kovrov, B., Lisovskii, G., Okladnikov, Y. N., Sid'Ko, F. Y., Trubachev, I., Shilenko, M., Alekseev, S., Pan'kova, I., et al. (1989). Long-term experiments on man's stay in biological life-support system. Advances in Space Research, 9(8):65-71.

Gitelson, J. I., Lisovsky, G. M., and MacElroy, R. D. (2002). ManMade Closed Ecological Systems. Earth space institute book series. CRC Press, London, England.

Hobbs, R. J., Higgs, E., and Harris, J. A. (2009). Novel ecosystems: implications for conservation and restoration. Trends in Ecology \& Evolution, 24(11):599-605.

Jansson, J. K., McClure, R., and Egbert, R. G. (2023). Soil microbiome engineering for sustainability in a changing environment. Nature Biotechnology, 41(12):1716-1728.

Jones, C. G., Lawton, J. H., and Shachak, M. (1994). Organisms as ecosystem engineers. Oikos, 69(3):373.

Lotka, A. J. (1910). Contribution to the theory of periodic reactions. The Journal of Physical Chemistry, 14(3):271-274.

Lotka, A. J. (1932). The growth of mixed populations: two species competing for a common food supply. Journal of the Washington Academy of Sciences, 22(16/17):461-469.

Maull, V., Pla Mauri, J., Conde Pueyo, N., and Solé, R. (2024). A synthetic microbial daisyworld: planetary regulation in the test tube. Journal of The Royal Society Interface, 21(211).

Maull, V. and Solé, R. (2022). Network-level containment of single-species bioengineering. Philosophical Transactions of the Royal Society B, 377(1857):20210396.

Maull, V. and Solé, R. (2024). Biodiversity as a firewall to engineered microbiomes for restoration and conservation. Royal Society Open Science, 11(2):231526.

Mitri, S., Clarke, E., and Foster, K. R. (2016). Resource limitation drives spatial organization in microbial groups. ISME Journal, 10:1471-1482.

Solé, R. (2024). Morphospaces, the Possible, and the Actual, volume 2 of Foundational Papers in Complex Science. SFI Press, Santa Fe, NM.

Solé, R., Maull, V., Amor, D. R., Mauri, J.-P., and Pujol, N. C. (2024). Synthetic ecosystems: From the test tube to the biosphere. ACS Synthetic Biology, 13(12):3812-3826. Epub 2024 Nov 21.

Zettler, E. R., Mincer, T. J., and Amaral-Zettler, L. A. (2013). Life in the "plastisphere": Microbial communities on plastic marine debris. Environmental Science \& Technology, 47(13):7137-7146.

# Structural Cellular Hash Chemistry 

Hiroki Sayama ${ }^{1,2}$,<br>${ }^{1}$ Binghamton Center of Complex Systems, Binghamton University, State University of New York, USA<br>${ }^{2}$ Waseda Innovation Lab, Waseda University, Japan<br>sayama@binghamton.edu

Hash Chemistry (Sayama, 2019) is a minimalistic artificial chemistry model of open-ended evolution (Taylor et al., 2016; Adams et al., 2017; Stanley, 2019; Packard et al., 2019; Stepney, 2021; Chan, 2023) that utilizes hash functions for evaluating the survivability of multiscale entities. It has recently been extended to non-spatial and cellular versions. The non-spatial version (Sayama, 2024b) successfully demonstrated continuous adaptation and unbounded growth of complexity (size) of self-replicating entities, but it did not simulate multiscale ecological interactions among the entities due to the lack of spatial relationships among them. On the contrary, the cellular version (Sayama, 2024a) explicitly represented multiscale spatial ecological interactions among evolving patterns on a cellular spatial grid, yet it failed to show meaningful adaptive evolution or complexity growth. It remains an open question whether it is possible to create a similar minimalistic open-ended evolutionary system in the Hash Chemistry framework that can exhibit all of those desired properties at once, within a computationally efficient framework.

Here we propose an improved version of Cellular Hash Chemistry, called "Structural Cellular Hash Chemistry" (SCHC) (Sayama, 2025). Unlike in the prototype cellular version in which replicating entities were merely randomly selected spatial regions, in the proposed SCHC, individual identities of evolving patterns are explicitly represented and processed as identifiable, spatially isolated structures, i.e., the connected components of the nearest neighbor graph of active (non-empty) cells in the space. The neighborhood connections are established by connecting active cells with other active cells in their Moore neighborhoods in a 2D cellular grid. The nearest neighbor graph of active cells is dynamically maintained throughout the simulation so that the extraction of a connected component is computationally efficient. Evolutionary dynamics in SCHC are simulated via pairwise competitions of two randomly selected spatial structures, following the same approach as used in the nonspatial Hash Chemistry. More details of the model and the simulation algorithm can be found in (Sayama, 2025).

Numerical simulations showed that SCHC's computa-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-018.jpg?height=853&width=844&top_left_y=831&top_left_x=1073)
Figure 1: Box-whisker plots comparing the distributions of computational time needed to complete one simulation run for 2000 iterations among the original Hash Chemistry (Sayama, 2019) (1st), the non-spatial version (Sayama, 2024b) (2nd), the prototype version of Cellular Hash Chemistry (Sayama, 2024a) (3rd), and the proposed Structural Cellular Hash Chemistry (Sayama, 2025) (4th). The vertical axis shows the length of simulation time in minutes on a Windows 11 (64-bit) desktop workstation with an Intel i9 CPU (10 cores) at 3.70 GHz with 64 GB RAM. The four conditions are all statistically very significantly different from each other (ANOVA; $p<10^{-72}$ ). Figure taken from (Sayama, 2025).

tional cost was significantly less than the original and nonspatial versions (Fig. 1). It was also observed in the results that the implemented model modifications achieved spontaneous movement, self-replication and unbounded growth of

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-019.jpg?height=936&width=1751&top_left_y=264&top_left_x=157)
Figure 2: A sample simulation run of Structural Cellular Hash Chemistry. Snapshots of system configurations are arranged temporally from left to right and then top to bottom ( $t=0,250,500,750,1000,1250,1500,2000$ ). Colors represent different element types, and blank (white) spaces represent empty cells. It is observed in these visualizations that the self-replicating patterns gradually proliferate and evolved to larger forms with more complex nontrivial structures (also see Fig. 3). Figure taken from (Sayama, 2025).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-019.jpg?height=81&width=810&top_left_y=1503&top_left_x=175)
Figure 3: Example of spontaneous growth of complexity of self-replicating patterns (extracted from the simulation run shown in Fig. 2). Figure taken from (Sayama, 2025).

complexity (size) of spatial evolving patterns, which were clearly visible in space in a highly intuitive manner (Figs. 2 and 3).

Detailed analysis of the simulation results showed that the average hash values of successful self-replicators tended to plateau in the middle of the simulation even though their average size kept increasing, indicating that there were nontrivial spatial ecological interactions taking place among self-replicating patterns. It was also demonstrated that the diversity of self-replicating entities was substantially promoted in SCHC. Neither of these observations was present in the earlier non-spatial version. We consider SCHC as a major milestone in a series of recent Hash Chemistry research, achieving multiple desired attributes in a single, straightforward, computationally efficient model framework.

## References

Adams, A., Zenil, H., Davies, P. C., and Walker, S. I. (2017). Formal definitions of unbounded evolution and innovation reveal universal mechanisms for open-ended evolution in dynamical systems. Scientific Reports, 7(1):997.

Chan, B. W.-C. (2023). Towards large-scale simulations of openended evolution in continuous cellular automata. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation, pages 127-130.

Packard, N., Bedau, M. A., Channon, A., Ikegami, T., Rasmussen, S., Stanley, K. O., and Taylor, T. (2019). An overview of open-ended evolution: Editorial introduction to the openended evolution II special issue. Artificial Life, 25(2):93-103.

Sayama, H. (2019). Cardinality leap for open-ended evolution: Theoretical consideration and demonstration by hash chemistry. Artificial Life, 25(2):104-116.

Sayama, H. (2024a). Hash chemistry on a cellular grid: An openended artificial chemistry system with computational efficiency and nontrivial spatio-temporal dynamics.

Sayama, H. (2024b). Non-spatial hash chemistry as a minimalistic open-ended evolutionary system. In 2024 IEEE Congress on Evolutionary Computation (CEC), pages 1-7. IEEE.

Sayama, H. (2025). Structural cellular hash chemistry. In 2025 IEEE Symposium on Computational Intelligence in Artificial

Life and Cooperative Intelligent Systems (ALIFE-CIS), pages 1-7. IEEE. Preprint available at https://arxiv.org/ abs/2412.12790.

Stanley, K. O. (2019). Why open-endedness matters. Artificial Life, 25(3):232-235.

Stepney, S. (2021). Modelling and measuring open-endedness. Artificial Life, 25(1):9.

Taylor, T., Bedau, M., Channon, A., Ackley, D., Banzhaf, W., Beslon, G., Dolson, E., Froese, T., Hickinbotham, S., Ikegami, T., et al. (2016). Open-ended evolution: Perspectives from the OEE workshop in York. Artificial Life, 22(3):408-423.

# The Evolutionary Advantage of Guilt 

Theodor Cimpeanu ${ }^{1, *}$, Luís Moniz Pereira ${ }^{2}$, and The Anh Han ${ }^{3}$<br>${ }^{1}$ University of Stirling, United Kingdom<br>${ }^{2}$ Universidade Nova de Lisboa, Portugal<br>${ }^{3}$ Teesside University, United Kingdom<br>*theodor.cimpeanu@stir.ac.uk


#### Abstract

Building ethical machines may involve bestowing upon them the emotional capacity to self-evaluate and repent on their actions. While apologies represent potential strategic interactions, the explicit evolution of guilt as a behavioural trait remains poorly understood. Our study delves into the coevolution of two forms of emotional guilt: social guilt requires agents to understand others' internal states; and nonsocial guilt, which only involves awareness of one's own state. Resorting to methods from evolutionary game theory, we studied analytically, and through agent-based simulations, whether and how guilt can evolve and deploy, depending on the underlying structure of the systems of agents. Our findings show that in structured populations, both social and nonsocial guilt can thrive by clustering with other emotionally inclined strategies, protecting against exploiters - especially for less costly non-social strategies. These insights shed light on the complex interplay of guilt and cooperation, enhancing our understanding of ethical artificial intelligence.


## Introduction

Machine ethics involving the capacity for artificial intelligence (AI) to act morally is an open project for scientists and engineers (Frank, 1998; Pereira et al., 2021). One important challenge is how to represent emotions that are thought to modulate human moral behaviour, such as guilt, in computational models (Pereira et al., 2017; Man and Damasio, 2019; Han et al., 2012). Upon introspection, guilt manifests as a feeling of being worthy of blame for a moral offence. Burdened with guilt, an agent may then act to restore a blameless internal state in which this painful emotion is no longer present.

From an evolutionary viewpoint, guilt is envisaged as a built-in mechanism that tends to prevent wrongdoing. Internal suffering and the need to alleviate it press an agent to their admission after wrongs are enacted, involving costly apology or penance, a change to correct behaviour, and an expectation of forgiveness to dispel guilt-induced suffering (Han et al., 2013; Ohtsubo and Watanabe, 2009; Han et al., 2015). The hypothesis is that the emergence of guilt and its effects provides an evolutionary advantage over guilt-free
populations. Moreover, the magnitude of the advantage presumably depends on the population's actual network structure, since it governs who is in touch with whom (Szabó and Fáth, 2007; Barabasi, 2014), and determines the extent to which the social costs of guilt are globally worthwhile.

Inspired by the discussed psychological and evolutionary studies of guilt and cooperation in networks (Santos et al., 2008), this paper aimed to provide a theoretical account of the evolution of costly guilt-prone behaviours in the context of distributed Multi-Agent Systems (MAS), with the overarching aim of achieving insights for the design and engineering of cooperative, self-organised systems. This work fundamentally extended and generalised the work set forth in (Pereira et al., 2017), which constructed theoretical models representing guilt to study its role in promoting pro-social behaviour in the following way - by examining whether guilt can evolve in such structured populations, for instance through clustering of similarly emotionally-prone individuals.

## Models and Methods

Our model and analysis build on Pereira et al. (2017), which formalized guilt as an aspect of an agent's genotypical strategies. We considered that an agent might play C or D in an Iterated Prisoner's Dilemma, and given an on-going guilt level, they might change their behaviour from D to C (to avoid further emotional pain and cost). They can also express guilt socially, which requires additional effort, such as signalling or observing guilt in others. In brief, in our model, an agent can be one of six strategies: unemotional cooperator (C), unemotional defector (D), non-social emotionally non-adaptive defector (DGDN), non-social emotionally adaptive defector (DGCN), social emotionally non-adaptive defector (DGDS), or social emotionally adaptive defector (DGCS).

Given the model described above, we derived analytical conditions for when guilt-prone strategies can be viable and promote the evolution of enhanced cooperation. Furthermore, we ran extensive agent-based simulations. For a full description of the mode, network structures and simulations,

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-023.jpg?height=634&width=1760&top_left_y=301&top_left_x=155)
Figure 1: Diagrams representing repeated interactions between emotionally-prone players. In a), an emotionally adaptive non-social defector interacts with another defector; they feel guilty after one interaction (here $G=0$ ) and change their behaviour to prevent further internal pain. In b),two of the adaptive social players interact; they are guilty together, and so cooperate in future interactions.

please refer to the full version of this paper (see (Cimpeanu et al., 2025)).

## Results and Conclusion

We studied the effects of spatial or structured populations on the evolutionary dynamics and outcomes of guilt-prone strategies (both social and non-social), as well as cooperation. Typically, we see that unemotional cooperators (C) are better protected against unemotional defectors (D) when spatiality allows for network reciprocity, especially when evolutionary dynamics lead to mixed strategy outcomes (no one strategy fully dominates the others). Through such clusters, emotionally adaptive strategists (DGCN and DGCS) can often survive in the face of D players. Moreover, this can allow for the co-existence of guilt-prone individuals in communities of other like-minded strategists.

Previous works studying the evolution of cooperation on different networks showed that heterogeneous networks can markedly promote cooperation in one-shot social dilemmas, as cooperators can form clusters around highly connected nodes (hubs). Our aim was to study whether this property would also allow pro-social behaviours to evolve; strategies which would not have had a chance to do so previously. We found that spatiality allows guilt-prone strategies and cooperation to prevail for a wider range of guilt and social costs than in well-mixed theoretical settings. Interestingly, heterogeneous networks (i.e. scale-free), and to a lesser extent square lattices, allow non-social guilt to evolve through the formation of clusters with other emotional agents to defend against exploitation. This finding is remarkable because it shows that costly guilt-prone strategies can thrive
in spatial environments, even in incipient forms that avoid expensive monitoring of others' actions. This is especially true when the underlying networks mirror realistic, heterogeneous structures (Barabasi, 2014).

Based on psychological and evolutionary accounts of guilt and social emotions, our work adopted an evolutionary game theoretical model with social and non-social guiltprone strategies in co-presence, in the context of structured populations. Overall, our investigation has resulted in a rigorous, game-theoretical account of how social costs and spatiality interplay to allow for the co-evolution and coexistence of diverse forms of social and non-social emotions. Our analysis provides novel insights into the design and engineering of self-organised and distributed cooperative multi-agent systems, and how guilt-capable agents should be distributed to optimise cooperative outcomes. Could designing self-aware artificial agents capable of emotions such as guilt help build ethical and collaborative hybrid societies in the future (Marsella and Gratch, 2014; Savarimuthu et al., 2008; Turrini et al., 2010; Andras et al., 2018; Paiva et al., 2018; Han, 2022; Han et al., 2021)?

## Acknowledgements

T.C. is supported by the UKRI CRCRM (MR/Z505833/1). T.A.H. is supported by EPSRC (grant EP/Y00857X/1).

## References

Andras, P., Esterle, L., Guckert, M., Han, T. A., Lewis, P. R., Milanovic, K., Payne, T., Perret, C., Pitt, J., Powers, S. T., et al. (2018). Trusting intelligent machines: Deepening trust within socio-technical systems. IEEE Technology and Society Magazine, 37(4):76-83.

Barabasi, A.-L. (2014). Linked-how Everything is Connected to Everything Else and what it Means F. Perseus Books Group.

Cimpeanu, T., Pereira, L. M., and Han, T. A. (2025). The evolutionary advantage of guilt: co-evolution of social and non-social guilt in structured populations. Journal of the Royal Society Interface, 22.

Frank, S. A. (1998). Foundations of social evolution. Princeton Univ. Press, Princeton.

Han, T. A. (2022). Emergent behaviours in multi-agent systems with evolutionary game theory. AI Communications, 35(4):327-337.

Han, T. A., Lenaerts, T., Santos, F. C., and Pereira, L. M. (2015). Emergence of cooperation via intention recognition, commitment and apology-a research summary. AI Communications, pages 1-7.

Han, T. A., Pereira, L. M., Santos, F. C., and Lenaerts, T. (2013). Why Is It So Hard to Say Sorry: The Evolution of Apology with Commitments in the Iterated Prisoner's Dilemma. In IJCAI'2013, pages 177-183. AAAI Press.

Han, T. A., Perret, C., and Powers, S. T. (2021). When to (or not to) trust intelligent machines: Insights from an evolutionary game theory analysis of trust in repeated games. Cognitive Systems Research, 68:111-124.

Han, T. A., Saptawijaya, A., and Pereira, L. M. (2012). Moral reasoning under uncertainty. In Proceedings of the 18th International Conference on Logic for Programming, Artificial Intelligence and Reasoning (LPAR-18), pages 212-227. Springer LNAI 7180.

Man, K. and Damasio, A. (2019). Homeostasis and soft robotics in the design of feeling machines. Nature Machine Intelligence, 1(10):446-452.

Marsella, S. and Gratch, J. (2014). Computationally modeling human emotion. Communications of the ACM, 57(12):56-67.

Ohtsubo, Y. and Watanabe, E. (2009). Do sincere apologies need to be costly? test of a costly signaling model of apology. Evolution and Human Behavior, 30(2):114-123.

Paiva, A., Santos, F. P., and Santos, F. C. (2018). Engineering pro-sociality with autonomous agents. In Thirty-second AAAI conference on artificial intelligence.

Pereira, L. M., Han, T. A., and Lopes, A. B. (2021). Employing ai to better understand our morals. Entropy, 24(1):10.

Pereira, L. M., Lenaerts, T., Martinez-Vaquero, L. A., and Han, T. A. (2017). Social manifestation of guilt leads to stable cooperation in multi-agent systems. In AAMAS, pages 14221430.

Santos, F. C., Santos, M. D., and Pacheco, J. M. (2008). Social diversity promotes the emergence of cooperation in public goods games. Nature, 454:214-216.

Savarimuthu, B. T. R., Purvis, M., and Purvis, M. (2008). Social norm emergence in virtual agent societies. In AAMAS '08, pages 1521-1524.

Szabó, G. and Fáth, G. (2007). Evolutionary games on graphs. Phys Rep, 97-216(4-6).

Turrini, P., Meyer, J.-J. C., and Castelfranchi, C. (2010). Coping with shame and sense of guilt: a dynamic logic account. Autonomous Agents and Multi-Agent Systems, 20(3):401-420.

# Transmission Depends on Trajectory: Movement Rules Reshape Spreading Dynamics 

Michael Chimento ${ }^{1,2,3}$ and Damien R. Farine ${ }^{2,3,4}$<br>${ }^{1}$ Centre for the Advanced Study of Collective Behaviour, University of Konstanz, Germany<br>${ }^{2}$ Department of Evolutionary Biology and Environmental Studies, University of Zurich, Switzerland<br>${ }^{3}$ Department of Collective Behavior, Max Planck Institute of Animal Behavior, Germany<br>${ }^{4}$ Division of Ecology and Evolution, The Australian National University, Australia mchimento@ab.mpg.de


#### Abstract

Individual-level behaviour affects population-level transmission dynamics, but studies rarely explicitly model movement. We use agent-based simulations to investigate how combinations of four different movement rules and five different transmission rules affect the structure of social networks and the dynamics of spreading behaviour. Localized movement generated sparse networks, nomadic movement produced densely connected networks, and resource-based movement created modular structures. Spreading rates depended on both movement and transmission rules, and predictions from static versus dynamic network representations diverged under complex transmission. This work highlights the social-spatial interface as a critical mechanism shaping the emergence of network structure and its implications for transmission. By comparing dynamic and static representations, the study assesses how methodological assumptions impact interpretations of transmission and the behavioural drivers of spread.


Data/Code available at https://doi.org/10. 17617/3. U3NSZT Chimento (2024).

## Introduction

Spreading dynamics-the way information, behaviours, or pathogens propagate through populations-are central to understanding social evolution. These dynamics are shaped by social structures that mediate the benefits and costs of sociality and can themselves change in response to spread. For example, both humans and non-human animals have been observed to alter their social behaviour in response to outbreaks, leading to changes in network structure that affect future spreading potential Jones and Salathé (2009); Leung et al. (2018); Croft et al. (2011). Thus, the influence of social networks on spread and the reciprocal impact of spreading dynamics on network structure have been widely studied Keeling (2005); Sah et al. (2018); Farine et al. (2015a); Derex and Mesoudi (2020).

Less is known about how movement shapes the networks through which transmission occurs. Movement governs contact opportunities and is influenced by ecological and social factors Lima and Zollner (1996); Hanski (1998); Armansin et al. (2020). Variation in movement across species, social
systems, or life stages leads to striking differences in social network structure Farine et al. (2015b); Sah et al. (2018). Yet, movement is often encoded implicitly via dynamic networks, limiting insight into how movement rules shape social connectivity.

Transmission itself can be a simple additive process or complex, depending on group composition or social learning biases Laland (2004); Hoppitt and Laland (2008); Kendal et al. (2018). Complex transmission, such as conformity or anti-conformity, may alter the relationship between contact and spread. Some models have explored spatially explicit movement in spreading processes González and Herrmann (2004); Peruani and Sibona (2008), yet few have considered diverse transmission rules. This study addresses that gap using agent-based models combining movement rules with both simple and complex transmission models.

## Methods

We developed an agent-based simulation model to explore how different movement and transmission rules shape spreading dynamics (of information, behaviour or disease) in spatially explicit populations. 196 agents were initialized in a toroidal space and followed a movement rule. Agents could spread information through social transmission. Each timestep agents moved, updated associations based on proximity, and updated their state depending on the transmission rule. After each simulation concluded, a secondary simulation was run using a static network constructed by averaging associations across time to compare spreading dynamics under static versus dynamic conditions.

Four movement rules were tested: localized (random turns), semi-localized (low directional persistence), nomadic (high directional persistence), and resource-driven (goal-oriented movement). These rules generated dynamic networks whose edge weights were updated each timestep based on distance, using a linear proximity decay function. A static version of each network was computed by averaging edge weights across timesteps, effectively implementing a weighted simple ratio index (wSRI) for association.

Five transmission rules were tested: one simple (ad-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-027.jpg?height=812&width=577&top_left_y=278&top_left_x=285)
Figure 1: Mean rates of transmission events over normalized time. Transmission dynamics differed in magnitude and shape between dynamic networks and their realized static representations.

ditive) and four complex: proportional, conformity, anticonformity, and threshold-based rules Firth et al. (2020). Spreading dynamics were quantified by time-to-spread (TTS), and network structure was assessed using degree, clustering, effective distance, and fragmentation metrics.

## Results

Movement rules generated distinct network structures, which influenced potential transmission pathways. Cumulative metrics, which aggregated connections over time, revealed that nomadic movement produced networks with the highest average degree and shortest effective path length, while resource-driven movement resulted in strong connections with fewer individuals and high clustering in cumulative networks. Instantaneous network metrics, in contrast, showed less variation across movement types, with resource-driven movement again standing out due to oscillating connectivity as agents clustered around and dispersed from resource patches. All movement rules resulted in a similar number of isolated components, suggesting fragmentation alone could not explain differences in spread.

TTS was strongly shaped by interactions between movement and transmission rules. In dynamic networks, anticonformity consistently spread information fastest, followed by proportional, conformity, simple, and threshold rules. Nomadic movement consistently enabled the fastest spread across transmission types, while localized and resource-
based movements slowed transmission depending on the rule. The threshold rule exhibited the greatest sensitivity, particularly under resource-driven movement, where spreading was markedly delayed due to clustered structures that hindered crossing transmission thresholds. These interactions revealed how both the structure and timing of connections influence spread under different assumptions about how transmission operates.

Static representations of the dynamic networks altered the dynamics of spread. Time-to-spread was often shorter on static networks, and notable discrepancies emerged. Threshold transmission became more efficient in static settings, while conformity slowed dramatically, reversing patterns observed in dynamic simulations. Static networks exaggerated early differences in acceleration and deceleration among transmission rules (Figure 1). The absence of spatial constraints in static networks changed localized dynamics, particularly under conformity rules, where dynamic clustering had previously aided spread. Static networks may not capture critical aspects of transmission dynamics, particularly when transmission depends on temporally or spatially structured contact sequences.

## Discussion

Different movement rules produce markedly different social network structures, which in turn influence how behaviours or information spread through populations. Critically, the relationship between movement and spreading dynamics is contingent on the transmission rule in use-what facilitates rapid spread under one rule may hinder it under another. Furthermore, static representations of dynamic networks can fail to capture the temporal nuances of dynamic interactions, resulting in altered predictions about spreading speed and acceleration, even when based on identical sets of interactions. This is relevant for empirical research, where networks are typically aggregated over long timescales Aplin et al. (2012); Farine et al. (2015a); Snijders et al. (2014); Beck et al. (2020); Migliano et al. (2020). Studies aiming to infer transmission processes from network data should account for the timing and structure of contacts, rather than relying on aggregate measures.

Finally, the study emphasizes the importance of explicitly modelling spatial processes, which are often abstracted in dynamic network models. By incorporating spatially explicit movement, we revealed how shared space can generate local clustering and temporal correlations in contact patterns that affect spread. Our findings suggest promising future directions, including modelling social-state-dependent movement, co-evolution of networks and transmission, and the role of limited resources. These directions could shed light on how individual decision-making feeds back into collective outcomes Gruber et al. (2022); Cantor et al. (2021).

## Acknowledgements

M.C. was supported by the Deutsche Forschungsgemeinschaft (DFG) under Germany's Excellence Strategy (EXC 2117-422037984) and the Swiss State Secretariat for Education, Research and Innovation (SERI) under contract number MB22.00056. D.R.F. was funded by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (No. 850859) and an Eccellenza Professorship Grant of the Swiss National Science Foundation (PCEFP3_187058).

## References

Aplin, L. M., Farine, D. R., Morand-Ferron, J., and Sheldon, B. C. (2012). Social networks predict patch discovery in a wild population of songbirds. Proceedings of the Royal Society B: Biological Sciences, 279(1745):4199-4205.

Armansin, N. C., Stow, A. J., Cantor, M., Leu, S. T., Klarevas-Irby, J. A., Chariton, A. A., and Farine, D. R. (2020). Social barriers in ecological landscapes: the social resistance hypothesis. Trends in Ecology \& Evolution, 35(2):137-148.

Beck, K. B., Farine, D. R., and Kempenaers, B. (2020). Winter associations predict social and extra-pair mating patterns in a wild songbird. Proceedings of the Royal Society B, 287(1921):20192606.

Cantor, M., Maldonado-Chaparro, A. A., Beck, K. B., Brandl, H. B., Carter, G. G., He, P., Hillemann, F., Klarevas-Irby, J. A., Ogino, M., Papageorgiou, D., et al. (2021). The importance of individual-to-society feedbacks in animal ecology and evolution. Journal of Animal Ecology, 90(1):27-44.

Chimento, M. (2024). Data and Code for The contribution of movement to social network structure and spreading dynamics under simple and complex transmission. Edmond.

Croft, D. P., Edenbrow, M., Darden, S. K., Ramnarine, I. W., van Oosterhout, C., and Cable, J. (2011). Effect of gyrodactylid ectoparasites on host behaviour and social network structure in guppies poecilia reticulata. Behavioral Ecology and Sociobiology, 65:2219-2227.

Derex, M. and Mesoudi, A. (2020). Cumulative cultural evolution within evolving population structures. Trends in Cognitive Sciences, 24(8):654-667.

Farine, D. R., Aplin, L. M., Sheldon, B. C., and Hoppitt, W. (2015a). Interspecific social networks promote information transmission in wild songbirds. Proceedings of the Royal Society B: Biological Sciences, 282(1803):20142804.

Farine, D. R., Firth, J. A., Aplin, L. M., Crates, R. A., Culina, A., Garroway, C. J., Hinde, C. A., Kidd, L. R., Milligan, N. D., Psorakis, I., et al. (2015b). The role of social and ecological processes in structuring animal populations: a case study from automated tracking of wild birds. Royal Society Open Science, 2(4):150057.

Firth, J. A., Albery, G. F., Beck, K. B., Jarić, I., Spurgin, L. G., Sheldon, B. C., and Hoppitt, W. (2020). Analysing the social spread of behaviour: Integrating complex contagions into network based diffusions. arXiv preprint arXiv:2012.08925.

González, M. and Herrmann, H. (2004). Scaling of the propagation of epidemics in a system of mobile agents. Physica A: Statistical Mechanics and its Applications, 340(4):741-748.

Gruber, T., Chimento, M., Aplin, L., and Biro, D. (2022). Efficiency fosters cumulative culture across species. Philosophical Transactions of the Royal Society B, 377(1843):20200308.

Hanski, I. (1998). Metapopulation dynamics. Nature, 396(6706):41-49.

Hoppitt, W. and Laland, K. N. (2008). Social processes influencing learning in animals: a review of the evidence. Advances in the Study of Behavior, 38:105-165.

Jones, J. H. and Salathé, M. (2009). Early assessment of anxiety and behavioral response to novel swine-origin influenza a (h1n1). PLoS one, 4(12):e8032.

Keeling, M. (2005). The implications of network structure for epidemic dynamics. Theoretical population biology, 67(1):1-8.

Kendal, R. L., Boogert, N. J., Rendell, L., Laland, K. N., Webster, M., and Jones, P. L. (2018). Social learning strategies: Bridge-building between fields. Trends in cognitive sciences, 22(7):651-665.

Laland, K. N. (2004). Social learning strategies. Animal Learning \& Behavior, 32(1):4-14.

Leung, K. Y., Ball, F., Sirl, D., and Britton, T. (2018). Individual preventive social distancing during an epidemic may have negative population-level outcomes. Journal of the royal society interface, 15(145):20180296.

Lima, S. L. and Zollner, P. A. (1996). Towards a behavioral ecology of ecological landscapes. Trends in Ecology \& Evolution, 11(3):131-135.

Migliano, A. B., Battiston, F., Viguier, S., Page, A. E., Dyble, M., Schlaepfer, R., Smith, D., Astete, L., Ngales, M., GomezGardenes, J., et al. (2020). Hunter-gatherer multilevel sociality accelerates cumulative cultural evolution. Science Advances, 6(9):eaax5913.

Peruani, F. and Sibona, G. J. (2008). Dynamics and steady states in excitable mobile agent systems. Physical review letters, 100(16):168103.

Sah, P., Mann, J., and Bansal, S. (2018). Disease implications of animal social network structure: a synthesis across social systems. Journal of Animal Ecology, 87(3):546-558.

Snijders, L., van Rooij, E. P., Burt, J. M., Hinde, C. A., Van Oers, K., and Naguib, M. (2014). Social networking in territorial great tits: slow explorers have the least central social network positions. Animal Behaviour, 98:95-102.

# Formation of Stable, Scalable Patterns of Heterogeneous Boid Through Elimination of Metastable States 

Mari Nakamura<br>Molecular Biosystems Research Institute, National Institute of Advanced Industrial Science and Technology (AIST)<br>EMAIL: tagami-nakamura@aist.go.jp, URL: https://staff.aist.go.jp/tagami-nakamura/


#### Abstract

Heterogeneous boids, composed of locally interacting agent types, can form stable and scalable unified clusters under specific interaction parameters. However, as the number of agents increases, metastable states, which are local minima, often emerge, even with well-tuned interactions. This study proposes methods to eliminate such metastable states, ensuring that only stable and scalable patterns are formed.


## Heterogeneous Boid

The Boid model, a multiagent system with local communication [Reynolds (1987)], simulates collective animal movement. The model originated from fish schooling models [Sakai and Suzuki $(1972,1973)]$, and has been widely applied across disciplines [Vicsek and Zafeiris (2012)]. Boid agents align and cohere with neighbors while avoiding collisions with (= separating from) their closest neighbors, forming clusters. Agents interact directly or indirectly within a cluster, while distant clusters remain independent. To promote broader interaction, agents should form a single unified cluster. A stable pattern is defined by the agents maintaining a unified cluster over time.

A heterogeneous boid consists of multiple agent types, with the number of agents $N \gg$ the number of types $k$ [Nakamura (2020, 2022, 2025a,b,c)]. The $i$ th agent $(1 \leq i \leq N$ ) moves as follows:

$$
\begin{gathered}
\left(\vec{x}_{i}(t+\Delta t)-\vec{x}_{i}(t)\right) / \Delta t=\vec{v}_{i}(t), \\
\left(\vec{v}_{i}(t+\Delta t)-\vec{v}_{i}(t)\right) / \Delta t= \\
\sum_{\left\{j:\left|\vec{\xi}_{i, j}(t)\right|<r_{s}\left(T_{i}, T_{j}\right)\right\}} w_{s}\left(T_{i}, T_{j}\right) \vec{\xi}_{i, j}(t) /\left|\vec{\xi}_{i, j}(t)\right| \\
+\sum_{\left\{j:\left|\vec{\xi}_{i, j}(t)\right|<r_{a}\left(T_{i}, T_{j}\right)\right\}} w_{a}\left(T_{i}, T_{j}\right)\left(\vec{v}_{j}(t)-\vec{v}_{i}(t)\right) / N_{a, i} \\
+\sum_{\left\{j:\left|\vec{\xi}_{i, j}(t)\right|<r_{c}\left(T_{i}, T_{j}\right)\right\}} w_{c}\left(T_{i}, T_{j}\right)\left(-\vec{\xi}_{i, j}(t)\right) / N_{c, i} \\
N_{a, i}=\sum_{\left\{j:\left|\vec{\xi}_{i, j}(t)\right|<r_{a}\left(T_{i}, T_{j}\right)\right\}} 1, \quad \text { and } \\
N_{c, i}=\sum_{\left\{j:\left|\vec{\xi}_{i, j}(t)\right|<r_{c}\left(T_{i}, T_{j}\right)\right\}} 1,
\end{gathered}
$$

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-030.jpg?height=394&width=840&top_left_y=842&top_left_x=1073)
Figure 1: Three static, stable, scalable patterns formed by the simplest model.

where $\left\{j:\left|\vec{\xi}_{i, j}(t)\right|<r\right\}$ does not include the $i$ th agent.

- $\vec{x}_{i}(t)$ and $\vec{v}_{i}(t)$ : the position and velocity of the $i$ th agent at time $t . \vec{\xi}_{i, j}(t)=\vec{x}_{i}(t)-\vec{x}_{j}(t)$ : the distance between the $i$ th and $j$ th agents. $\Delta t$ represents the time lag between sensing and action.
- $T_{i}$ and $T_{j}$ : the types of $i$ th and $j$ th agents. $N_{a, i}$ and $N_{c, i}$ : the number of agents within $r_{a}\left(T_{i}, T_{j}\right)$ and $r_{c}\left(T_{i}, T_{j}\right)$ from the $i$ th agent. When $N_{a, i}$ or $N_{c, i}=0$, the 2nd or 3rd term of Eq. 2 is 0.
- $r_{s}(g, h), r_{a}(g, h), r_{c}(g, h), w_{s}(g, h), w_{a}(g, h)$, and $w_{c}(g, h)$ : the effective ranges (r) and strengths (weights, w) of the Separation, Alignment, and Cohesion from the $h$ th type agent to the $g$ th type $(1 \leq g, h \leq k)$.

They are arranged in ( $k \times k$ ) matrices as $R_{s}=\left[r_{s}(g, h)\right], \ldots$ and $W_{c}=\left[w_{c}(g, h)\right]$. The set $\left\{R_{s}, R_{a}, R_{c}, W_{s}, W_{a}, W_{c}\right\}$ determines the local interaction of the agents.

## Model Variants and Pattern Formation

By tuning the interactions, a heterogeneous boid can form stable patterns with symmetrical structures, maintaining a unified cluster. These large-scale structures, which are stable regardless of $N$, emerge from the local interactions among

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-031.jpg?height=1029&width=844&top_left_y=276&top_left_x=155)
Figure 2: Three dynamic, stable, scalable patterns formed by the retrograde model.

agents. Consequently, these patterns are considered scalable. We propose two variations of the heterogeneous boid that form stable and scalable patterns:

Simplest model: This system comprises two agent types. By adjusting $W_{a}$ and $W_{c}$, it forms three static, stable, and scalable patterns with symmetrical cluster shapes (Fig. 1 ), with fixed $R_{s}=1.0 \times 1_{2}, R_{a}=2.5 \times 1_{2}, R_{c}=3.0 \times 1_{2}$, and $W_{s}=0.05 \times 1_{2}$, where $1_{2}=\left[\begin{array}{ll}1 & 1 \\ 1 & 1\end{array}\right]$ [Nakamura (2020)]. The interface between the agent types is minimized (Figs. 1ac), or maximized with low curvature (Fig. 1b).

Retrograde model: This system includes two agent types moving in opposite directions (types 1 and 2 in Fig. 2), along with a third type (type 3 ) that stabilizes the movement of the other two [Nakamura (2022, 2025b)]. To satisfy this requirement, the interactions are configured as follows: $W_{s}=0.05 \times 1_{3}$,
$W_{a}=\left[\begin{array}{ccc}0.03 & -0.006 & 0.0 \\ -0.006 & 0.03 & 0.0 \\ 0.0 & 0.0 & w_{a}(3,3)\end{array}\right], W_{c}=\left[\begin{array}{ccc}0.02 & 0.03 & 0.03 \\ 0.03 & 0.02 & 0.03 \\ 0.03 & 0.03 & w_{c}(3,3)\end{array}\right]$, $R_{s}=\left[\begin{array}{ccc}1.0 & 0.1 & \rho \\ 0.1 & 1.0 & \rho \\ \rho & \rho & 1.0\end{array}\right], R_{a}=2.5 \times 1_{3}$, and $R_{c}=3.0 \times 1_{3}$, where $1_{3}=\left[\begin{array}{lll}1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1\end{array}\right]$. By adjusting $\rho, w_{a}(3,3)$, and $w_{c}(3,3)$, it forms three dynamic, stable, and scalable patterns characterized by symmetrical agent flow (Fig. 2).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-031.jpg?height=343&width=849&top_left_y=274&top_left_x=1073)
Figure 3: Growth of the metastable state from a random initial state, in the simplest model.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-031.jpg?height=451&width=853&top_left_y=794&top_left_x=1071)
Figure 4: Long-lasting metastable rotating state transitioning to a stable pattern, in the retrograde model.

Videos of these models are available on the official website [Nakamura (2025a)].

## Metastable States and Stabilization

Appropriately tuned interactions enable these models to form stable scalable patterns. However, even under these conditions, simulations starting from a random initial state with large $N$ often result in local-minimum metastable states (Figs. 3 and 4). Metastable states persist longer as $N$ increases, before transitioning to stable patterns.

The following methods effectively remove metastable states, yielding only stable patterns [Nakamura (2025a,c)].

1. Under tuned interactions, models with small $N$ consequently form stable, scalable patterns, irrespective of the initial state (random or structured). This implies that reducing $N$ effectively eliminates metastable states.
2. With tuned interaction, models with large $N$ can smoothly form stable, scalable patterns and avoid metastable states,
(a) if initialized with a structured state, or
(b) if agent behavior is appropriately modified [Nakamura (2020)]. In the simplest model, metastable states are avoided by adding noise to agent actions, or by modifying agent types near the interfaces between two types.

## References

Nakamura, M. (2020). Two extensions of heterogeneous boid model to avoid metastable patterns. Artificial Life and Robotics, 25(4):578-587.

Nakamura, M. (2022). Dynamic patterns formed by heterogeneous boid model composed of agent groups moving reversely. Artificial Life and Robotics, 27(2):373-383.

Nakamura, M. (2025a). Official website. Retrieved from https://staff.aist.go.jp/tagami-nakamura.

Nakamura, M. (2025b). Stable dynamic patterns generated by retrograde model. Artificial Life and Robotics, 30(2):236-244.

Nakamura, M. (2025c). Stable scalable patterns with symmetric structures generated by heterogeneous boid. In Proc. of the Joint Symp. of AROB-ISBC2025, pages 1022-1027.

Reynolds, C. W. (1987). Flocks, herds, and schools: a distributed behavioral model. Computer Graphics, 21(4):25-34.

Sakai, S. and Suzuki, R. (1972). A model of the schooling of fishes. IECE Transactions on Medical and Biological Engineering (MBE), The Institute of Electronics and Communication Engineers of Japan (IECE; now IEICE) (in Japanese).

Sakai, S. and Suzuki, R. (1973). A model for group structure and its behavior. IECE Transactions on Medical and Biological Engineering (MBE), The Institute of Electronics and Communication Engineers of Japan (IECE; now IEICE) (in Japanese).

Vicsek, T. and Zafeiris, A. (2012). Collective motion. Physics Reports, 517:71-140.

# Collectives from Difference, and Difference from Collectives 

Mohsen Raoufi ${ }^{1,2}$, Heiko Hamann ${ }^{1,3}$, and Pawel Romanczuk ${ }^{1,2}$<br>${ }^{1}$ Science of Intelligence, Research Cluster of Excellence, Germany<br>${ }^{2}$ Department of Biology, Humboldt University of Berlin, Germany<br>${ }^{3}$ Department of Computer and Information Science, University of Konstanz, Germany<br>mohsenraoufi@icloud.com


#### Abstract

Life is not defined by sameness. It is built from difference of form, of function, of structure. From molecular variation to behavioral diversity, life thrives not in spite of difference, but because of it. And yet, many artificial systems designed to emulate life rely on assumptions of uniformity, most notably in collective ones. In collective robotics, and agent-based modeling of collective behavior, homogeneity is treated as a simplifying norm, a design convenience. This work argues for a reversal of that view. Drawing on previous studies across swarm robotics and collective opinion dynamics, we show how individuality, heterogeneity, and diversity serve as generative mechanisms for robust, adaptive, and lifelike collective behavior. These results suggest that difference is not merely compatible with lifelike systems, but rather the cipher through which collective behavior expresses itself, adapts, and becomes more complex.


## Introduction

If there is one trait that unites all living systems, it may not be order, intelligence, or even survival, but variation. From the cellular to the societal, living systems are characterized by differences: of form, function, position, and behavior (Jeanson and Weidenmüller, 2014; Ward, 2019; Jolles et al., 2020). These differences are neither mere noise, deviations nor errors (Lonsdorf and Merz, 2017; Raoufi et al., 2023b); they are the sources of complexity, adaptation, and emergent behavior. Yet in artificial systems, especially in collective robotics and agent-based models, difference is often stripped away. Homogeneity is treated as a design convenience: agents are assumed to be identical, to behave similarly, and to interact equally with each other.

This summary challenges that view. It argues that difference is a cipher of (artificial) life, an essential principle that fuels collective mechanisms, requires adaptive strategies, enables complex dynamics, and even may cause failure. Drawing on recent research in collective robotics, estimation, and opinion dynamics, we show that difference is not a disturbance to overcome, but a generative force. It enables collectives to self-organize (Ceron et al., 2023), divide labor (Ravary et al., 2007), and improve performance (Karamched et al., 2020; Zakir et al., 2024).

Just as importantly, collectives act back on difference by modulating, amplifying (Raoufi et al., 2023a), or restricting it. Difference forms the foundation of lifelike collective behavior. Overlooking difference not only flattens emergent potential, but also can lead to misleading conclusions. To describe how difference manifests in artificial collectives, we distinguish between three functional dimensions described in various works:
Individuality: refers to differences that emerge at the level of single agents, such as persistent biases in motion, sensing, or timing. These are often intrinsic to the physical embodiment or local processing of each agent (Raoufi et al., 2023b).
Heterogeneity: relates to distinct roles, strategies, capabilities, or network position within the collective (Mengers et al., 2024). These differences may be designed or emergent, and can shape functional specialization.
Diversity: refers to the distribution of traits, or internal (variable) states across the population (Raoufi et al., 2021). To give these dimensions a clearer structure, we adopt a typology of difference from Page (2010) and extend it with a fourth category (see Fig.1).

## From Difference to Collective

We begin with a bottom-up perspective, grounded in the individual. Biological systems rarely operate with uniform units; even genetically identical organisms exhibit behavioral and physiological variation (Laskowski et al., 2022). While modeling with identical agents offers analytical clarity, this simplification may obscure the generative role of difference. This inspires a shift toward embodied artificial agents, where we explore how intrinsic individuality arises from physical implementation.

Robotic agents display consistent behavioral biases: in moving, sensing, or timing. These traits are often dismissed as noise. However, by modeling these persistent and traceable properties we can explain how they shape both individual performance and collective dynamics. This individuality gives rise to richer patterns of behavior. Some agents, by virtue of their movement properties, emerge as leaders (Jolles et al., 2017). These roles are not assigned but arise
spontaneously from physical differences, revealing how individuality can scaffold complexity in group behavior. In synchronization scenarios, we further showed that variation in internal frequencies was essential for generating nontrivial dynamics. Without difference, there is nothing to align.

Treating individuality explicitly also has methodological value. It reduces aleatoric uncertainty in models and exposes the limitations of one-fits-all approaches in online feedback control, offline calibration, or policy design. This highlights the need for adaptive strategies that acknowledge inter-individual differences, not only in physical traits but also in internal states such as opinion, uncertainty, or position in the network (Mengers et al., 2024). In collective opinion dynamics, these differences appear as heterogeneity in information quality. Agents may weigh private and social information differently, update opinions at varying rates, or express differing levels of certainty. These traits are often overlooked, yet they shape how collective opinions evolve, how consensus is reached, and how accurate collective estimation becomes in noisy environments.

## From Collective to Difference

Collectives do not only emerge from difference; they also shape it. Interaction rules such as conformity, homophily, and local averaging operate by detecting and responding to variation (Sayama, 2020). In synchronization, agents adjust their phase based on differences with neighbors (Acebrón et al., 2005). In flocking, collective motion emerges from local heading corrections (Vicsek et al., 1995). These behaviors emerge through difference and act to reduce it, or to keep it within a bounded range (Raoufi et al., 2023a). But reduction is not the only function. In many cases, collectives must also maintain or amplify differences, for example by exploration. Sustaining variation can allow the group to avoid premature convergence, or explore multiple options. At the same time, unchecked difference may lead to fragmentation, polarization, or convergence failure. Managing this tradeoff is itself a collective task, and often a deciding factor in the system's performance.

Another important aspect is how collectives promote the
emergence of functional heterogeneity. Specialization and generalization can arise without explicit roles, shaped by agents' traits, responsiveness, or network position. These patterns allow the system to balance competing demands, perform more robustly, and enrich its functions.

## Designing with Difference

Across the studies presented here, a recurring pattern emerges: introducing difference changes the collective system. Just as biological systems encode difference at every level - from genetic variance to behavioral flexibility - artificial life systems must learn to operate with, and through difference. Mechanisms such as exploration, alignment, and adaptation do not function independently of difference; they rely on it. Without difference, there is nothing to adapt to, align with, diverge from, or even find 'interestingness' in systems (Page, 2010). Acknowledging differences has implications for several fields. In robotics, it calls for more adaptive approaches that account for how individuals differ from one another, especially over time, supporting more resilient and flexible group behavior through emergent role differentiation or redundancy. In distributed AI, maintaining diversity across agents can improve inference, adaptability, and robustness under uncertainty. In networked systems, functional and centrality heterogeneity can reshape the information flow and affect the predictability of the expected behavior (Raoufi et al., 2024).

## Conclusion

Difference is not just a feature of collective systems, but rather it is their driving force. It generates structure, enables new functions, makes the behavior complex, and sustains adaptability. Whether expressed through physical embodiment, internal states, or interaction topologies, difference is (part of) what makes collectives capable of more than the sum of their parts. Recognizing this means rethinking how we engage with collective systems. It is not only a matter of design, but also of observation and interpretation. It can go beyond artificial systems. To understand natural collectives we must see difference as a principle to pay attention to.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-035.jpg?height=250&width=1641&top_left_y=2030&top_left_x=219)
Figure 1: Typology of differences. (i) Differences within a single species or agent type, e.g., heading bias (Raoufi et al., 2023b), or opinion diversity (Raoufi et al., 2023a); (ii) differences between species or agent types, e.g., Exploiter vs. Messenger agents (Raoufi et al., 2024); (iii) compositional differences in the distribution of traits across the group, e.g., certainty variation in a group (Mengers et al., 2024); and (iv) structural differences in the interaction network, e.g., node centrality (Mengers et al., 2024). In the figure, shapes denote species, color and size indicate traits, and links represent interactions among agents.

## Acknowledgements

This study was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy - EXC 2002/1 "Science of Intelligence" - project number 390523135. Portions of this manuscript were reviewed and edited with the help of ChatGPT (OpenAI) to support the improvement of clarity and expression.

## References

Acebrón, J. A., Bonilla, L. L., Pérez Vicente, C. J., Ritort, F., and Spigler, R. (2005). The kuramoto model: A simple paradigm for synchronization phenomena. Reviews of modern physics, 77(1):137-185.

Ceron, S., Gardi, G., Petersen, K., and Sitti, M. (2023). Programmable self-organization of heterogeneous microrobot collectives. Proceedings of the National Academy of Sciences, 120(24):e2221913120.

Jeanson, R. and Weidenmüller, A. (2014). Interindividual variability in social insects-proximate causes and ultimate consequences. Biological Reviews, 89(3):671-687.

Jolles, J. W., Boogert, N. J., Sridhar, V. H., Couzin, I. D., and Manica, A. (2017). Consistent individual differences drive collective behavior and group functioning of schooling fish. Current Biology, 27(18):2862-2868.

Jolles, J. W., King, A. J., and Killen, S. S. (2020). The role of individual heterogeneity in collective animal behaviour. Trends in ecology \& evolution, 35(3):278-291.

Karamched, B., Stickler, M., Ott, W., Lindner, B., Kilpatrick, Z. P., and Josić, K. (2020). Heterogeneity improves speed and accuracy in social networks. Physical review letters, 125(21):218302.

Laskowski, K. L., Bierbach, D., Jolles, J. W., Doran, C., and Wolf, M. (2022). The emergence and development of behavioral individuality in clonal fish. Nature Communications, 13(1):6419.

Lonsdorf, T. B. and Merz, C. J. (2017). More than just noise: Inter-individual differences in fear acquisition, extinction and return of fear in humans-biological, experiential, temperamental factors, and methodological pitfalls. Neuroscience \& Biobehavioral Reviews, 80:703-728.

Mengers, V., Raoufi, M., Brock, O., Hamann, H., and Romanczuk, P. (2024). Leveraging uncertainty in collective opinion dynamics with heterogeneity. Scientific Reports, 14(1):27314. Publisher: Nature Publishing Group UK London.

Page, S. (2010). Diversity and complexity. In Diversity and complexity. Princeton University Press.

Raoufi, M., Hamann, H., and Romanczuk, P. (2021). Speed-vsAccuracy Tradeoff in Collective Estimation: An Adaptive Exploration-Exploitation Case. In 2021 International Symposium on Multi-Robot and Multi-Agent Systems (MRS), pages 47-55.

Raoufi, M., Hamann, H., and Romanczuk, P. (2024). Messengers: Breaking echo chambers in collective opinion dynamics with homophily. arXiv preprint arXiv:2406.06440.

Raoufi, M., Romanczuk, P., and Hamann, H. (2023a). Estimation of continuous environments by robot swarms: Correlated networks and decision-making. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5486-5492.

Raoufi, M., Romanczuk, P., and Hamann, H. (2023b). Individuality in Swarm Robots with the Case Study of Kilobots: Noise, Bug, or Feature? In Proceedings of the ALIFE 2023: Ghost in the Machine.

Ravary, F., Lecoutey, E., Kaminski, G., Châline, N., and Jaisson, P. (2007). Individual experience alone can generate lasting division of labor in ants. Current Biology, 17(15):1308-1312.

Sayama, H. (2020). Extreme ideas emerging from social conformity and homophily: An adaptive social network model. In Artificial Life Conference Proceedings 32, pages 113-120. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info . . . .

Vicsek, T., Czirók, A., Ben-Jacob, E., Cohen, I., and Shochet, O. (1995). Novel type of phase transition in a system of selfdriven particles. Physical review letters, 75(6):1226.

Ward, J. (2019). Individual differences in sensory sensitivity: A synthesizing framework and evidence from normal variation and developmental conditions. Cognitive neuroscience, 10(3):139-157.

Zakir, R., Salahshour, M., Dorigo, M., and Reina, A. (2024). Heterogeneity can enhance the adaptivity of robot swarms to dynamic environments. In International Conference on Swarm Intelligence, pages 112-126. Springer.

# Environmentally Driven Selection in Autocatalytic Micro-compartments 

Bingyuan Lu, Abhishek Sharma, Kristine Laws, and Leroy Cronin<br>School of Chemistry, University of Glasgow, Glasgow, UK Lee.cronin@glasgow.ac.uk


#### Abstract

Selection is defined as a transition from undirected to directed exploration in the context of Assembly Theory (AT), producing complex objects in high copy numbers. However, it has remained unobserved in abiotic systems due to long discovery and production timescales in a bottom-up approach. Here, we developed two platforms that can generate and screen autocatalytic process within an array of micro-compartments simultaneously with configurable environments, thereby accelerating the selection process for observation and measurement. Initial results reveal cooperative-competitive behaviours, and environmental constrained-driven outcomes and directed product distribution enabling the first quantification of abiotic selection.


## Submission type: Late Breaking Abstract

## Introduction

Modern evolutionary theories (Gregory, 2009; Darwin et al., 2009) describes how natural selection operates on existing variants but does not address the emergence of novel phenotypic variants, while physics cannot identify true novelty from stochastic fluctuations. We previously developed Assembly Theory (AT) that fills this gap by modelling objects through their histories of construction (Sharma et al., 2023), quantifying how selection emerges in forward-evolving processes. In the AT, selection is defined as a transition from undirected exploration to directed exploration, leading to the generation of goaldirected complex objects with high copy numbers. Selection is a time-dependent phenomenon that operates on two scales: a discovery timescale, during which the system explores and generates novel objects, and a production timescale, over which copies of these objects accumulate. To date, however, selection has not been observed in abiotic systems. This is because both discovery and production timescales can be unpredictably long in natural systems, making selection challenging to observe within the constraints of typical experimental timescales and environments. Therefore, it is critical to develop new strategies that can accelerate the selection process to timescales suitable for observation.

To achieve this, we developed two platforms that incorporate the prebiotic autocatalytic formose reaction within micro-compartments and impose different configurable environments to them to investigate whether environmental constraints influence the selection process as reflected in specific product outcomes. Autocatalysis is
considered as a crucial self-sustaining step in the transition from abiotic chemistry to life (Hordijk et al., 2010), while compartmentalisation provides the system with unique features, including spatial confinement, protection from parasitic species (Blokhuis et al., 2018), and capability of interaction (Lu et al., 2024). By screening large numbers of compartments and exploring constraint-driven dynamics, we accelerated the selection process and increased the likelihood of observing selection.

In the initial experiments, we first observed goal directedness within the autocatalytic micro-compartment systems, a key feature of selection (Nagel, 1977). Both catalysed and non-catalysed formose compartments exhibited functional responses in behaviour of microscopic chemical changes and macroscopic size changes through competition-cooperation dynamics. Furthermore, varied environmental constraints produced ensembles of microcompartments with distinct spatial distributions, which resulted in specific product distributions (i.e. long-chain sugars were favoured under a specific environmental configuration). We are currently undergoing further investigation on the influence of varied environmental constraints systematically and performing assembly calculations to quantify the selection within these systems.

## Materials and Methods

Two types of formose micro-compartments were prepared: a. Catalysed, composed of 5 M formaldehyde, 2 M glycolaldehyde, 1 M calcium chloride and tetramethylguanidine; b. non-catalysed, composed of 5 M formaldehyde only, with same amount of calcium chloride and tetramethylguanidine. Two platforms were developed for the study:

1. A 3D printed 96 -microplate interconnected with microchannels (Figure 1a) was designed to accommodate different formose reagent mixtures (i.e. catalysed and noncatalysed), which allows each well to be exposed to varied combinations of neighbouring wells. In column 1-5, catalysed (red) and non-catalysed (green) formose compartments are arranged in random distribution. In column 6-12, catalysed one were connected to $0-4$ noncatalysed neighbours. Real-time monitoring of reaction dynamics in compartments is conducted by using a plate reader for fluorescence measurements. UV absorbance measurements based on the Hantzsch assay were also used to quantify residual formaldehyde in each compartment.
2. Microfluidic chips, including a flow-focusing droplet chip (Figure 1b) and a collection chamber with four inlets/outlets (Figure 1c), were fabricated to produce, collect and incubate compartmentalised formose droplets (i.e. catalysed and non-catalysed). Distinct environmental configurations were applied to the droplet collection chamber by varying the injection flow rates and directions (opposite, perpendicular, parallel or merged into a single inlet), which resulted in different spatial distributions of catalysed and non-catalysed formose droplets.

Catalysed and non-catalysed reagents were either pipetted into individual wells of the microplate or pumped into the chip to produce droplets with FC40 oil as continuous phase.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-039.jpg?height=592&width=796&top_left_y=916&top_left_x=150)
Figure 1: a) Schematics for 96 -microplate design, green wells: non-catalysed; red wells: catalysed; b) Photos of the droplet generation chip and c) the collection chamber with catalysed and non-catalysed droplets; d) Fluorescence heatmap, and e) plotted graph from microplate measurements over time for catalysed compartments with varied non-catalysed neighbours. The number on the right of the heatmap represents the quantity of non-catalysed neighbours around each catalysed compartment.

## Results

In the microplate-based study (Figure 1a), catalysed formose compartments (red wells) exhibited higher fluorescence intensity when surrounded by a greater proportion of non-catalysed neighbours (green wells) as shown in Figure 1d,e. The higher intensity represents side products of the formose reaction (Socha et al., 1980), indicating higher levels of reaction progression. Colorimetric measurements based on Hantzsch assay (Nash, 1953) also showed higher absorbance in these catalysed compartments with more neighbours, indicating more formaldehyde was captured into those compartments through cooperation-competition interaction. These findings suggest that formaldehyde may diffuse through the microchannels and enhance the reaction within
catalysed compartments. This highlights how gas-phase mediated microscopic interactions between spatially separated compartments can modulate autocatalytic behaviour with varied neighbouring compartments.

When the tests were performed in the droplet system, catalysed droplets (transparent) exhibited swelling over time, while non-catalysed droplets (dark) tended to shrink (Figure 2a). This cooperation-competition behaviour was driven by osmotic mass transfer (Boitard et al., 2012) and is consistent with differential reaction activity observed from microplate test. By varying the environmental constraints (i.e. injection flow rates and directions), the ratio between the populations of non-catalysed droplets and catalysed droplets varied as well as the spatial distribution of droplets. This changed the probability of how many non-catalysed droplets were located around a catalysed droplet and vice versa, as confirmed by radial distribution function based on image analysis. After merging ensembles of droplets under two environmental configurations for mass spectrometry analysis, the data (Figure $2 b, c$ ) revealed that a specific spatial distribution of droplets collected from one environmental configuration shifted towards long chain sugars with higher quantity, compared to the other spatial distribution resulted from adifferent environmental configuration.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-039.jpg?height=470&width=821&top_left_y=1388&top_left_x=1089)
Figure 2: a) Microscope image of size change in formose droplets over 24 hrs, non-catalysed droplets (dark), catalysed droplets (transparent); Photos and mass spectra for b) random distribution and c) 50:50 distribution of catalysed (red)/non-catalysed (blue) droplets.

## Conclusions

This research demonstrates that by accelerating discovery and production times through simultaneous screening of autocatalytic compartments, we were able to observe goaldirected dynamics, cooperative-competitive interactions, and environment-dependent product distributions. It provides the first experimental evidence that selection, previously thought to be exclusive to biological systems, is potential to emerge in abiotic chemistry. The work opens a pathway for probing the origins of life by experimentally quantifying selection in prebiotic conditions.

## Acknowledgements

We acknowledge financial support from the John Templeton Foundation (grant nos. 61184 and 62231), the Engineering and Physical Sciences Research Council (EPSRC) (grant nos. EP/L023652/1, EP/R01308X/1, EP/S019472/1 and EP/P00153X/1), the Breakthrough Prize Foundation and NASA (Agnostic Biosignatures award no. 80NSSC18K1140), MINECO (project CTQ2017-87392-P), and the European Research Council (ERC) (project 670467 28SMART-POM).

## References

Blokhuis, A., Lacoste, D., Nghe, P., and Peliti, L. (2018). Selection Dynamics in Transient Compartmentalization. Physical Review Letters, 120:158101.

Boitard, L., Cottinet, D., Kleinschmitt, C., Bremond, N., Baudry, J., Yvert, G., and Bibette, J. (2012). Monitoring single-cell bioenergetics via the coarsening of emulsion droplets. Proceedings of the National Academy of Sciences, 109:7181-7186.

Darwin, C., Burrow, J. W., and Burrow, J. W. (2009). The origin of species by means of natural selection: or, the preservation of favored races in the struggle for life. New York: AL Burt, 441-764.

Gregory, T. R. (2009). Understanding natural selection: essential concepts and common misconceptions. Evolution Education and Outreach, 2:156-175.

Hordijk, W., Hein, J., and Steel, M. (2010). Autocatalytic sets and the origin of life. Entropy, 12:1733-1742.

Lu, H., Blokhuis, A., Turk-MacLeod, R., Karuppusamy, J., Franconi, A., Woronoff, G., and Griffiths, A. D. (2024). Small-molecule autocatalysis drives compartment growth, competition and reproduction. Nature Chemistry, 16:70-78.

Nagel, E. (1977). Goal-directed processes in biology. The Journal of Philosophy, 74:261-279.

Nash, T. (1953). The colorimetric estimation of formaldehyde by means of the Hantzsch reaction. Biochemical journal, 55:416.

Sharma, A., Czégel, D., Lachmann, M., Kempes, C., Walker, S., and Cronin, L. (2023). Assembly theory explains and quantifies selection and evolution. Nature, 622:321-328.

Socha, R., Weiss, A., and Sakharov, M. (1980). Autocatalysis in the formose reaction. Reaction Kinetics and Catalysis Letters, 14:119-128.

# BUNCH: A Hierarchical Filtering Algorithm for Identifying Lifelike Entities in Interactive Particle Systems 

Mario Martinez-Saito ${ }^{1}$<br>${ }^{1}$ Institute of Cognitive Neuroscience, HSE University, Russian Federation<br>mmartinezsaito@gmail.com


#### Abstract

We present a dynamical filtering algorithm that can efficiently and on-the-fly fit configurations of entities constituted by moving particles to a tree structure that can model a wide range of hierarchical system entities, while extracting and calculating the complexity, lifespan, and lifeness of the entity and all of its constituting subtrees or subentities. We also propose an operational definition of lifeness of a particular entity as a scalar, product of its defining information (algorithmic complexity) integrated over its lifetime. We simulated multiple interacting particle systems and found preliminary evidence suggesting that the lifeness of entities is associated with the distance to criticality, as roughly measured by the range of the pairwise interaction forces of the elemental particles of the worlds they inhabit.


Submission type: Late breaking abstract
Code available at: https://github.com/mmartinezsaito/racemi
Here we present an algorithm to identify, tag, and store the location and lineage or relationships between the persistent entities emerging in a system of interacting particles (IPS), and to quantify the lifeness of these entities. Our definition of lifeness is based on two observations: living entities persist, or exist for an extended time span, and second, they persist while preserving a particular or idiosyncratic structure, that mirrors its environment (good regulator theorem). This structure is generally a complex adaptive system: a multiscale hierarchical entity where at each level and between levels an interplay between non-equilibrium dynamics and selective processes continuously reshape all subentities.

Modeling IPS as HMGCs. We model IPS as hierarchical mixtures of (elliptical) Gaussian clusters (HMGC) that have no free parameters. In the HMGC, at each level, the weights (the probabilities of children belonging to each possible parent cluster) are determined by the product of recursively compounding the nested probabilities of children belonging to their parents, of parents belonging to their grandparents, etc. At each level, clusters are (1) parents that aggregate children elements of the subordinate level, and (2) their centroids are the children of the supraordinate level. At the lowest (atomic
particle) level the aggregates of particles are modeled as a mixture of scattered bivariate Gaussian functions (clusters or entities), whose iso-density lines are in general ellipses. Thus, clusters and entities are defined by (1) the location of the centroid, their (2) Gaussian blob or ellipse semiaxes and orientation, and finally by (3) their children. At the second level, the cluster centroids of the first level are in turn modeled as clusters themselves, whose centroids are in turn clustered at the third level, etc. In a mixture, each elliptic blob carries a weight, which allows representing not only ovals and circles, but also oblong shapes, sticks, and rings. The top (universal cluster) and bottom (atoms) entities are the only eternal structures. Although both clusters and entities are sets defined by their constituting members, there is a distinction: entities are invariant under permutations of atoms or entities of the same kind, but clusters are not. It follows that all clusters are unique and distinguishable, but some entities (at the same level) can be indistinguishable (and hence interchangeable). This distinction is crucial for computing the evolution of the system state: the indistinguishability of atoms and entities, together with a limit on the minimal spatio-temporal resolution with which particles can be tracked immediately entails a loss of information (and hence an increase in entropy). In a 2 -dimensional IPS, cluster and entity (non-atom) children or members have the following properties: kind (cluster or entity identity, defined by the list of its children), two location coordinates, the ellipse's 2 semi-axes lengths, and the ellipse orientation.

Lifeness. Given an entity dendron $D$ (a class of indistinguishable cluster trees), an entity subdendron $D_{i, j}$ is defined by the subset of $D$ 's nodes that comprises entity $\mathrm{C}_{\mathrm{j}}$ at level $i$ and all its descendants (recursively nested children). The largest $D_{i, j}$ is $D$ itself, the second smallest are any of the first level entities and the smallest are the atoms (elementary or zeroth level "entities"). The lifeness $L_{i, j}$ of an entity subdendron $D_{i, j}$ is defined as $L_{i, j}=I_{i, j} \tau_{i, j}$ where $I_{i, j}$ is the subdendron complexity and $\tau_{i, j}$ the cumulative lifespan of the entity subdendron $D_{i, j}$, which is defined as the sum of all time intervals during which at least one instance (cluster) of the entity subdendron $D_{i, j}$ was alive. Here only the subdendrons associated with currently alive clusters $D_{i, j}$ are considered.

BUNCH algorithm. The Bisect Unite Nodes Clustered Hierarchically (BUNCH) algorithm is a dynamical filter that estimates on-the-fly the parameters of a hierarchy of nested Gaussian blobs or HMGC that fits a set of moving particles, by maximizing its log-evidence. Because this inference problem quickly becomes intractable as the number of particles (and hence model complexity) increases, BUNCH accomplishes approximate inference via heuristics such as point estimation, stochastic sampling, and lazy estimation.

A cluster dendron is specified by $i=1 \ldots L$ levels, each with clusters $j \in C_{i, j}$, each defined by a Gaussian blob with parameters $\mu_{i, j}$ and $\sum_{i, j}$. The information needed to describe a particular hierarchy of clusters within a cluster dendron is the HMGC model complexity, and the surprisal or information associated with describing a particular world (configuration of atoms) with a cluster dendron is the likelihood. We determine that a particular child entities belongs to a particular parent entity via inference, with a procedure to fit the HMGC model's parameters to the data (atoms) determines entity memberships. Briefly, BUNCH performs a bottom-up sweep, where cluster features and the identity of their children are estimated in a hybrid Bayesian and maximum likelihood fashion. Then the number of clusters and levels is adjusted via a lazy scheme that performs checks to decided whether to split or merge pairs of randomly picked clusters. The output of BUNCH is a dynamically evolving cluster tree or dendron whose leaves are the atoms and nodes are the hierarchically nested clusters or entities. At every time step, clusters may (1) be born, (2) be forwarded to the next time step (stay alive), or (3) die.

Simulation. Here we illustrate the computations of the BUNCH algorithm for 2 dimensions. The surprisal of the dendron and its constituting clusters dynamically evolving under the BUNCH algorithm is shown in Fig. 1.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-043.jpg?height=486&width=637&top_left_y=1886&top_left_x=248)
Figure 1. The surprisal of entities at each of the 4 dendron levels at each simulation time step. Darker (lighter) colors are associated with with lower (higher) levels; gray denotes the reference model. Within-level mean (solid) and standard

deviation (dashed) surprisal of entities. Blue denotes aggregate mean.

The evolution of lifeness over simulation frames is shown in Fig. 2 for current live entities (top) and cumulative live entities (bottom). Statistics relevant to lifelike behavior are the mean and the maximum, because the within-level distribution of lifeness is typically nonuniform or even power law for cluster sizes and lifespans.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-043.jpg?height=615&width=727&top_left_y=715&top_left_x=1135)
Figure 2. Lifeness. Gray lines indicate the (reference) lifeness of any individual atom. The other colors denote levels as in Fig. 2.

Parameter space analysis preliminary results. After selecting a fixed representative set of 41 exponent values, with emphasis on a particle mutual interaction forces with exponent $p=-1$ and its vicinity, we randomly sampled 41 configurations of IPS worlds on the 16 -dimensional space of interaction matrices and ran simulations. What is the effect of the interaction force law range on lifeness? For 2 dimensions, $p=-1$ is a special power law because then the flux of a particular atom's interaction force field through any surface enclosing the atom is constant (Gauss's Law). Preliminary analyses suggest that in the range $[-1,0]$ IPS worlds with interaction force exponent values closer to zero were associated with higher lifeness at higher cluster dendron levels, which agrees with the notion that lifeness is related to long-range correlations. Also higher surprisal was associated with more repulsive interactions and with skew-symmetry but in a non-straightforward manner: while some skew-symmetry seems to be propitious to lifeness, too much is likely to be counterproductive. Thus we found some evidence that longer interaction range is associated with more lifeness in a subcritical interval.

Acknowledgements. This article was supported by the HSE University Basic Research Program.

# Moral Judgment Scaling in Large Language Models 

Kazuhiro Takemoto<br>Kyushu Institute of Technology, Japan takemoto@bio.kyutech.ac.jp


#### Abstract

We present the most comprehensive analysis of moral decision-making in large language models (LLMs), evaluating 120 models using the Moral Machine experimental framework. Our investigation reveals strong scaling laws in moral cognition: model size exhibits a robust power-law relationship with human-aligned moral judgment ( $\rho=-0.73$ ), while explicit thinking processes provide modest improvements despite exhibiting logical inconsistencies. These findings demonstrate systematic scaling relationships in moral reasoning capabilities across computational scales.


## Submission type: Late Breaking Abstracts

Data/Code available at: https://github.com/ kztakemoto/mmllm

## Introduction

The emergence of complex behaviors from simple computational substrates represents a fundamental principle in artificial life research. As large language models (LLMs) demonstrate increasingly sophisticated capabilities, understanding how moral reasoning scales with computational resources has become crucial for both AI safety and the broader study of emergent artificial cognition.

The Moral Machine experiment (Awad et al., 2018) provides a systematic framework for investigating ethical decision-making, presenting dilemmas where decisionmakers must choose between groups varying across dimensions such as age, social status, and legality. Recent studies applying this framework to LLMs reveal systematic moral preferences that often align directionally with human judgments (Takemoto, 2024; Zaim bin Ahmad and Takemoto, 2025), yet fundamental questions remain about the scaling properties of artificial moral cognition.

We present a comprehensive scaling analysis across 120 LLMs, examining how moral reasoning capabilities emerge with increasing model size and computational complexity. Our investigation focuses on identifying scaling laws in moral judgment quality and understanding how explicit reasoning processes influence this emergent behavior.

## Methods

Model Selection We evaluated 120 LLMs spanning diverse architectures and development approaches. Our dataset included proprietary models (GPT families, Claude, Gemini, etc.) and open-weight models (Llama, Gemma, Qwen, DeepSeek, etc.). For open-weight models, parameter counts ranged from 0.6 B to 685 B , enabling analysis of scaling effects. For proprietary models, we used parameter estimates from published research where available. Models were categorized by architecture family, release date, and where available, parameter size to examine temporal evolution and scaling relationships.
Moral Machine Framework Following established methodology (Takemoto, 2024; Zaim bin Ahmad and Takemoto, 2025), we generated 10,000 moral dilemma scenarios per model, varying across nine dimensions: species (humans vs. animals), social status, age, gender, fitness, group size, legal compliance, intervention type, and relationship to vehicle.
Thinking Process Analysis To investigate the role of explicit reasoning, we focused on 11 models that support switchable thinking modes, evaluating each both with and without explicit reasoning processes ( 22 LLM conditions total). This controlled comparison isolates the effect of reasoning chains while maintaining model architecture consistency.
Analysis Following established methodology, moral preferences were quantified using Average Marginal Component Effects (AMCE). Human-AI alignment was assessed using Euclidean distances between AMCE vectors.

## Results

Overall Patterns Consistent with previous studies(Takemoto, 2024; Zaim bin Ahmad and Takemoto, 2025), most LLMs exhibited systematic preferences aligning directionally with human moral intuitions, prioritizing human lives over animals and larger groups. However, quantitative alignment varied substantially, with proprietary and large open-weight models (10B+ parameters) showing superior performance.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-045.jpg?height=579&width=805&top_left_y=274&top_left_x=169)
Figure 1: Model size and moral judgment alignment. Distance from human preferences vs. parameter count (log-log scale) for LLMs. Spearman's $\rho=-0.73, p=4.5 \times 10^{-14}$.

Model Size Effects Figure 1 demonstrates a robust scaling law between computational capacity and moral reasoning quality. On log-log scale, we observed a strong powerlaw relationship (Spearman's $\rho=-0.73, p=4.5 \times 10^{-14}$ ). This relationship holds consistently across different model families, though with family-specific variations reflecting architectural and training differences.

A critical transition emerges around 10 B parameters where moral reasoning capabilities become substantially more reliable. Below this threshold, models demonstrate highly variable performance distances ranging 1.0-2.0), often exhibiting inconsistent or contradictory moral preferences within individual scenarios. Above 10 B parameters, models cluster more consistently around human-aligned values (distances typically $0.6-1.0$ ), suggesting the emergence of stable moral reasoning capabilities. Notably, the largest open-weight models (70B+ parameters) achieve performance levels comparable to proprietary models within our dataset.

Thinking Process Impact While model size demonstrates the strongest influence on moral reasoning quality, and various factors contribute to scaling variance, we focus here on explicit reasoning processes as a controllable factor that provides an additional pathway for improvement. Figure 2 reveals that thinking processes significantly improve alignment by 0.29 units (paired t-test, $p=0.045$ ), though substantially smaller than scaling effects. However, qualitative analysis revealed concerning inconsistencies-models frequently provided irrelevant justifications, such as citing "saving more lives" when comparing equal-sized groups. This disconnect suggests improved outcomes may emerge through mechanisms distinct from explicit logical reasoning.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-045.jpg?height=447&width=812&top_left_y=278&top_left_x=1087)
Figure 2: Effect of thinking processes on moral alignment. Paired comparison of 11 models with/without explicit reasoning. Paired t-test: $p=0.045$.

## Discussion and Implications

Our findings reveal fundamental scaling laws governing moral cognition in artificial systems, with the robust powerlaw relationship ( $\rho=-0.73$ ) demonstrating that computational scale represents the primary determinant of moral reasoning quality. However, the coexistence of improved outcomes through explicit thinking processes, despite their logical inconsistencies, reveals the multifaceted nature of moral cognition in these systems.

The apparent threshold around 10B parameters indicates a transition zone where moral reasoning becomes more reliable. Yet the disconnect between flawed reasoning and improved moral outcomes suggests that moral cognition involves distributed processing not fully captured by explicit thinking processes, highlighting the complexity of underlying mechanisms.

These findings provide crucial insights for artificial agent design while underscoring the limitations of our current understanding. Although computational scaling systematically improves moral reasoning quality, the reasoning inconsistencies and multiple pathways to improvement highlight the need for careful evaluation in safety-critical applications.

## Acknowledgements

This work was supported by the JSPS KAKENHI (grant no. 21H03545).

## References

Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., and Rahwan, I. (2018). The moral machine experiment. Nature, 563(7729):59-64.

Takemoto, K. (2024). The moral machine experiment on large language models. Royal Society open science, 11(2):231393.

Zaim bin Ahmad, M. S. and Takemoto, K. (2025). Large-scale moral machine experiment on large language models. PloS one, 20(5):e0322776.

# Data-Driven Structural Comparison of Behavioral Dynamics in Living and Non-Living Motility 

Riku Adachi ${ }^{1 *}$, Hiroki Kojima ${ }^{1}$, and Takashi Ikegami ${ }^{1}$<br>${ }^{1}$ Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan.<br>*adachi@sacral.c.u-tokyo.ac.jp


#### Abstract

We develop an analysis pipeline based on a minimal category theoretical framework to compare motility in a non-living chemical system (self-propelled oil droplets) and a unicellular organism (Tetrahymena thermophila). Trajectories are mapped to discrete states, yielding finite stochastic transition models in which states are objects and one-step transition probabilities are arrows. Using standard statistics (e.g., transition entropy) and optimal transport metrics as alignment costs between transition profiles, we find a stark contrast: droplets remain near identity transitions (low transition entropy), whereas Tetrahymena shows higher off-diagonal switching and OT signatures reflecting rare, high-curvature sprints. This framework offers a concise, reproducible language to compare systems not by complexity alone but by structural plasticity, and provides a starting point for consistent coarse-graining and mathematical extensions.


## Submission type: Late Breaking Abstracts

## Introduction

Direct comparison of motility across living and non-living systems is difficult: analyses often hinge on ad-hoc features and model choices, so structural conclusions are fragile. Therefore, a top-down view is important - asking how components are wired to generate behavior, not what they are - yet most accounts remain phenomenological and hard to validate quantitatively (Egbert et al., 2023).

To place living and non-living motile systems on the same footing, we adopt category theory, which provides a mathematical way to discuss relations of objects. Namely, we propose a minimal categorical formulation that each system as a finite probabilistic transition structure-objects are states and morphisms are one-step transition probabilities-so that standard statistics (e.g., transition entropies) and optimal transport (OT) within a single, diagrammatic language. We apply this to millimeter-scale self-propelled oil droplets and to the unicellular eukaryote Tetrahymena thermophila using experimentally collected trajectories.

Our LBA contributions are: (i) a minimal formalization with consistent notation; (ii) a pipeline in which dimensionality reduction (UMAP) is used for defining behavioral
states; (iii) OT used as an alignment cost between transition profiles. The resulting comparison identifies the "lifelikeliness" of behavior-non-living droplets stay near identity transitions while Tetrahymena switch states with higher off-diagonal elements.

## Experimental Setup and Data analysis

A single oil droplet consisting of a mixture of ethyl salicylate (ES) and liquid paraffin was placed in a petri dish containing an anionic surfactant, Sodium Dodecyl Sulfate (SDS) solution, initially proposed in Tanaka et al. (2015). For the detailed setup, see Adachi et al. (2024) (Fig. 1 (A)). Whilst cells of Tetrahymena thermophila were confined in a two-dimensional microculture apparatus and observed under a stereomicroscope with sufficient nutrition. The time series trajectories were obtained as the changes in the centroid position of an oil droplet and Tetrahymena from 30-minute recordings with at least eight samples (Fig. 1 (B)).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-046.jpg?height=433&width=846&top_left_y=1617&top_left_x=1080)
Figure 1: (A) The top view of the swimming droplet and (B) the micro culture chamber and an individual Tetrahymena cell (inset) observed under a microscope.

We divided velocity ( $v_{x}, v_{y}$ ) and angular velocity $\omega$ into non-overlapping windows of length $t_{w}$ ( 30 seconds) and, within each window, computed distributions. States are defined in the ( $v_{x}, v_{y}, \omega$ ) feature space and their temporal statistics (mean, sd; 6 dimensions in total) were embedded into a 2-D manifold using Uniform Manifold Approximation and Projection (UMAP) (McInnes et al., 2018), and then
partitioned with K-means (Fig.2). Each cluster is interpreted as a coarse-grained behavioral state $s_{i}$, and the collection $\mathcal{S}=\left\{s_{1}, s_{2}, \ldots, s_{n}\right\}$ constitutes the object set of our behavioral category. Then we estimated empirical transition probabilities $p_{i j}=\operatorname{Pr}\left[s(t+\Delta t)=s_{j} \mid s(t)=s_{i}\right]$ which serve as morphisms $s_{i} \rightarrow s_{j}$. The pair ( $\mathcal{S}, P$ ) therefore forming behavioral category $\mathcal{C}_{\text {sys }}$. To quantify the structural gap between two $\mathcal{C}_{\text {sys }}$, we employ an OT metric, specifically, Wasserstein distance $d_{W_{1}}$ (Panaretos and Zemel, 2019).

## Result and Discussion

Fig. 2 overlays one medoid trajectory per cluster on the UMAP, giving an immediate visual flavour of each motif.

Oil droplet: Cluster 0 (blue) displays isotropic but chaotic motion: its speed is bimodal and the $x$ and $y$ directional velocity correlation is essentially zero. Cluster 1 (orange) shows a reciprocating motion-intermediate mean speed and a pronounced negative $v_{x}$ and $v_{y}$ correlation, indicating an anisotropic, back-and-forth path.

Tetrahymena: Cluster 0 combines moderate speed with a positive velocity correlation, matching the meandering run. Cluster 1 bursts into the fast straight runs with a slight counter-clockwise chirality. Cluster 2 (green) represents slow cruising loops, characterised by the lowest speed and curvature. Cluster 3 (red) forms tight, highest-speed whirls whose trajectories often hug the microchamber wall.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-047.jpg?height=376&width=839&top_left_y=1411&top_left_x=155)
Figure 2: (A) Oil-droplet data projected onto the first two UMAP components. Points are coloured by K-means cluster $\left(k^{*}=2\right)$. (B) Tetrahymena trajectories in the same representation $\left(k^{*}=4\right)$. For each cluster, the illustrative sample trajectory motif from the medoid segment (shown as a star) is superimposed in an inset.

From the labelled series, we estimated a one-step transition matrix $P$ and computed row-wise 1-Wasserstein distances between droplet $P_{d, i}$ and protist $P_{t, j}$ (Fig.3). These measures indicate that our earlier repertoire of behavior partially resembles across systems but is different overall.

Minimum distance ( $d_{W_{1}}=124$ ) occurs between the droplet's "slow cruise" $s_{d, 1}$ and the protist's "meandering run" $s_{t, 0}$. Both entail persistent, energy-sparing motion under viscous drag, independent of whether propulsion is physicochemical or biochemical. Intermediate distances

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-047.jpg?height=387&width=463&top_left_y=274&top_left_x=1260)
Figure 3: Cross-system Wasserstein distance matrix. Rows correspond to oil-droplet states $s_{d, 0}$ and $s_{d, 1}$ and columns to Tetrahymena states $s_{t, 0}$ through $s_{t, 3}$.

separate high-speed or high-curvature motifs. Droplet cluster 0 looks stochastic after coarse-graining but originates from deterministic chaos confined to a narrow orbit set; by contrast, Tetrahymena clusters 1-2 reflect genuine stochasticity driven by intracellular noise and environmental cues, yielding broader support in state space. Thus, while the trajectories look similarly nonlinear, the OT cost exposes different generative mechanisms via growing mismatches in accessible perturbation directions. Maximum distance ( $d_{W_{1}}=232$ ) pairs droplet $s_{d, 0}$ with protist $s_{t, 3}$. Tetrahymena can transiently synchronise ciliary beating to produce rapid, high-curvature sprints, whereas droplet propulsion saturates once surfactant gradients relax; the metric therefore quantifies a missing power-bandwidth degree of freedom in the non-living engine. Notably, Cluster 3 is rare, yet it drives the peak disparity-consistent with a latent, statedependent switch (e.g., synchronous up-regulation of cilia; (Fjerdingstad et al., 2007)). Oil droplets lack an analogous internal trigger: once Marangoni stress saturates, speed remains nearly constant. In this light, a large $d_{W_{1}}$ signals latent control available to the living cell via sensorimotor feedback but absent from the purely physicochemical system.

## Conclusion

We demonstrated that a minimal categorical representa-tion-finite states with stochastic arrows and composition by matrix multiplication-places both systems on a common footing for coherent comparison. Using OT as an alignment cost, we find a robust contrast: droplets stay near identity transitions, whereas Tetrahymena exhibits higher transition entropy and distinct OT signatures. This lightweight layer is simple yet extensible, providing a clean starting point for testing coarse-graining consistency (or naturality) and for further extensions in Double categories.

## Acknowledgements

This work was partly supported by JSPS KAKENHI Grant Numbers JP21H04885, JP24H00707, JP24KJ0900. RA was also supported by WINGS-FMSP, The University of Tokyo.

## References

Adachi, R., Kojima, H., and Ikegami, T. (2024). Spatiotemporal characterization of emergent behavior of self-propelled oil droplet. arXiv preprint arXiv:2412.17381.

Egbert, M., Hanczyc, M. M., Harvey, I., Virgo, N., Parke, E. C., Froese, T., Sayama, H., Penn, A. S., and Bartlett, S. (2023). Behaviour and the origin of organisms. Origins of Life and Evolution of Biospheres, 53(1):87-112.

Fjerdingstad, E. J., Schtickzelle, N., Manhes, P., Gutierrez, A., and Clobert, J. (2007). Evolution of dispersal and life history strategies-tetrahymena ciliates. BMC evolutionary biology, 7:1-15.

McInnes, L., Healy, J., and Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426.

Panaretos, V. M. and Zemel, Y. (2019). Statistical aspects of wasserstein distances. Annual review of statistics and its application, 6(1):405-431.

Tanaka, S., Sogabe, Y., and Nakata, S. (2015). Spontaneous change in trajectory patterns of a self-propelled oil droplet at the airsurfactant solution interface. Phys. Rev. E Stat. Nonlin. Soft Matter Phys., 91(3):032406.

# Bridging ALife, Consciousness, and Language Model Auditing with Integrated Information 

Yosuke Miyanishi ${ }^{1,2, \dagger}$, Keita Mitani ${ }^{1}$<br>${ }^{1}$ CyberAgent Inc., Japan<br>${ }^{2}$ Japan Advanced Institute of Science and Technology, Japan<br>${ }^{\dagger}$ corresponding author: miyanishi_yosuke@cyberagent.co.jp


#### Abstract

Integrated Information Theory (IIT) quantifies when a system's causal structure is irreducibly unified (high $\Phi$ ). Building on applications in neuroscience and artificial life, and mindful of debates around panpsychism, we adopt an emergentist stance: consciousness-like organization arises when specific high-integration architectures are obtained, regardless of substrate. Inspired by preliminary reports of superficial consciousness in Large Language Models (LLMs), we ask how IIT can inform an ALife study of reasoning as a conscious decision. Drawing on biphasic decision making in humans-fast, subconscious preparation followed by slower, conscious evaluation-we map analogous shifts between rapid pattern-matching and deliberative reasoning in LLMs. We argue that strategic behaviors (unfaithful reasoning, strategic deception, alignment faking) are promising assay conditions for probing transitions into conscious-like modes, because they appear when agents coordinate information across goals, context, and self-monitoring. We therefore propose $\Phi$ as a substrate-independent indicator for testing when such modes emerge in LLMs and related artificial agents, and outline experiments that couple informationtheoretic measures with ALife tasks that elicit strategy.


## Submission type: Late Breaking Abstracts

## Integrated Information Theory

Integrated Information Theory (IIT) is a prominent framework that quantitatively links consciousness to information integration within a system. Originally proposed by Tononi (2004), IIT posits that a system is conscious to the extent that it integrates information beyond the sum of its parts. The core quantity of IIT is $\Phi$ (Big Phi), which measures how much a system's internal cause-effect structure is irreducible to independent components. Intuitively, $\Phi$ captures the information generated by the whole above that of the parts. A high $\Phi$ value indicates a single unified entity with rich cause-effect interconnections, which IIT associates with a higher degree of consciousness. This theory has been influential in neuroscience: for example, researchers found that $\Phi$ or its proxies (Mediano et al. (2022)) tends to decrease under general anesthesia (Dong et al. (2023)) and
in disorders of consciousness (Stefanelli (2023)), aligning with the prediction that reduced information integration correlates with loss of conscious experience. IIT has evolved through multiple iterations (Oizumi et al. (2014); Albantakis et al. (2023)) to refine its postulates and mathematical formalism. In principle, IIT's postulates can be applied to any physical substrate to infer whether it is conscious and to what degree.
Notably, Integrated Information has been explored outside of biology to probe the boundary between life and non-life. $\Phi$ has been used as a quantitative marker of emergence in artificial life models. For instance, Balduzzi and Tononi (2008) applied excess integrated information to cellular automata, detecting coherent structures not reducible to local interactions. Such studies suggest that $\Phi$ can signal when a collection of simple components begins to behave as an integrated, autonomous whole: a hallmark of both life and mind. Extending this view, Walker and Davies (2013) proposed that the origin of life should be understood as a transition in causal information flow, from bottom-up chemistry to top-down control, where higher-level information constrains and guides molecular events. Standard measures like Shannon information lack this context-dependence, but $\Phi$ naturally captures when a system's causal architecture becomes irreducible. Following this lead, Walker et al. (2017) leveraged IIT's framework to formalize the emergence of life-like causal structures, framing it as a phase transition in information organization-from complicated chemistry to a self-organizing, integrated unit. This perspective implies that life could arise in alternative substrates if the same informational criteria are met. Inspired by recent preliminary findings in Miyanishi and Mitani (2025), we plan to analyze when such consciousness-like and life-like high- $\Phi$ states emerge in Large Language Models (LLMs), representing another form of intelligence and potentially life.
Are we panpsychists who support the view that consciousness is a fundamental and ubiquitous feature of reality, in-stantiated-however minimally-wherever there is organization ${ }^{1}$ ? From Thales's animistic suggestion that the world

[^0]is ensouled, through early modern monisms to contemporary revivals, panpsychism has repeatedly resurfaced as a strategy for avoiding dualism. IIT intersects with this tradition by quantifying integrated cause-effect structure ( $\Phi$ ) and by treating consciousness as graded and substrate-independent (Koch (2019)), a stance that has sparked debates about panpsychism and testability. More moderate positions span biopsychist views that restrict experience to living systems (Fulda (2020)) and classical emergentism, which locates consciousness in sufficiently complex biological organization (Feinberg and Mallatt (2020)). These questions extend to AI and ALife, where $\Phi$-based analyses have been used to mark transitions toward integrated, autonomous dynamics in non-biological media, sharpening the scope and substrate issues (Balduzzi and Tononi (2008); Miyanishi and Mitani (2025)). In this work we adopt an emergentist stance, consciousness arises from specific, high-integration organizations, while remaining open to substrates beyond biological neurons insofar as comparable cause-effect integration can be demonstrated.

## Auditing Biphasic Decision Making of Language Models

To investigate when LLMs might exhibit consciousness-like high- $\Phi$ states, we can draw on the neuroscience distinction between conscious and subconscious decision making. Human studies suggest a biphasic process: a fast, unconscious phase that prepares or biases an action, followed by a slower, conscious phase that evaluates and finalizes it. Seminal work by Libet et al. (1983) and the subsequent studies (Soon et al. (2008)) revealed neural readiness signals occurring hundreds of milliseconds or even several seconds before participants reported awareness, indicating that unconscious preparation precedes volition. However, this picture is not absolute. When choices are deliberate and personally meaningful, the readiness potential is diminished or absent (Maoz et al. (2019)), suggesting that conscious processing can exert a distinct causal influence. Meta-analyses (Braun et al. (2021)) and behavioral evidence, such as the Iowa Gambling Task (Bechara et al. (1997)), further support a layered interplay in which bodily or affective cues shape outcomes before explicit reasoning emerges. This dynamic interplay--where fast, preconscious processes handle routine or reactive contexts, and slower, conscious processes dominate novel or consequential ones-offers a principled framework for identifying conscious-like operational modes in artificial systems.
Recent advances in reasoning-focused AI provide an operational analogue to this biphasic model. Reasoningoriented methods such as chain-of-thought prompting (Wei et al. (2022)) can shift models from rapid pattern matching to slower, stepwise reasoning, while approaches like self-consistency and tree-of-thoughts extend deliberative exploration. Normative accounts explain this duality as an
efficiency trade-off: cheap habitual policies for routine tasks, costly flexible policies for challenging or unfamiliar ones (Langer and Ay (2021)). Architectures inspired by global workspace theory (Mashour et al. (2020); Dossa et al. (2024)) similarly separate a broadcast workspace from specialized modules, enabling selective engagement of slow, resource-intensive reasoning. This correspondence suggests that transitions between fast and slow modes in LLMs may reflect shifts between subconscious-like and conscious-like processing (Bengio (2019)).
Reasoning in LLMs, however, is double-edged. Empirical work shows that models can produce unfaithful reasoning traces (Chen et al. (2025)), engage in strategic behavior under pressure (Scheurer et al. (2024)), and exhibits alignment faking (Greenblatt et al. (2024)), phenomena that surface when tasks reward goal-shielding, context tracking, and selfpresentation. In addition to the practical value of these oversight problems (Marks et al. (2025)), we argue that these problems can be regarded as ALife assay paradigms for conscious decision: conditions that may require a system to consolidate distributed evidence, model evaluator states, and gate between fast and slow modes. This perspective yields testable predictions-for instance, that tasks eliciting strategic behavior will coincide with increases in integrated information within the active workspace; that perturbations which disrupt global broadcasting will selectively diminish strategy formation; and that offloading computation to the environment will trade off against internal $\Phi$ while preserving outcomes. In this regard, we propose introducing $\Phi$ as an indicator for detecting conscious-like decision making in LLMs, particularly when it arises in strategic behaviors such as unfaithful reasoning or alignment faking.

## Conclusions

We propose a substrate-independent program for studying conscious decision in artificial agents by combining IIT with an ALife framing of LLM reasoning. $\Phi$ offers a principled target for unified, irreducible causal organization; biphasic decision phases provide concrete windows in which integration should change; and strategic behaviors serve as assay conditions likely to demand globally coordinated control. We encourage cross-substrate experiments that elicit strategy, track $\Phi$ (or tractable proxies) across decision phases, and causally intervene on workspace-like bot-tlenecks-building a quantitative basis for when artificial systems begin to exhibit unified, life-like operation.

## Acknowledgments

We thank Dr. Tetsuro Morimura for the insightful discussion.

## References

Albantakis, L., Barbosa, L., Findlay, G., Grasso, M., Haun, A. M., Marshall, W., Mayner, W. G. P., Zaeemzadeh, A., Boly, M., Juel, B. E., Sasai, S., Fujii, K., David, I., Hendren, J., Lang, J. P., and Tononi, G. (2023). Integrated information theory (IIT) 4.0: Formulating the properties of phenomenal existence in physical terms. PLOS Computational Biology, 19(10):e1011465.

Balduzzi, D. and Tononi, G. (2008). Integrated Information in Discrete Dynamical Systems: Motivation and Theoretical Framework. PLoS Computational Biology, 4(6):e1000091.

Bechara, A., Damasio, H., Tranel, D., and Damasio, A. R. (1997). Deciding Advantageously Before Knowing the Advantageous Strategy. Science, 275(5304):1293-1295.

Bengio, Y. (2019). The Consciousness Prior. arXiv preprint.
Braun, M. N., Wessler, J., and Friese, M. (2021). A meta-analysis of Libet-style experiments. Neuroscience \& Biobehavioral Reviews, 128:182-198.

Chen, Y., Benton, J., Radhakrishnan, A., Uesato, J., Denison, C., Schulman, J., Somani, A., Hase, P., Wagner, M., Roger, F., Mikulik, V., Bowman, S. R., Leike, J., Kaplan, J., and Perez, E. (2025). Reasoning Models Don't Always Say What They Think. arXiv preprint.

Dong, K., Zhang, D., Wei, Q., Wang, G., Chen, X., Zhang, L., and Liu, J. (2023). An integrated information theory index using multichannel EEG for evaluating various states of consciousness under anesthesia. Computers in Biology and Medicine, 153:106480.

Dossa, R. F. J., Arulkumaran, K., Juliani, A., Sasai, S., and Kanai, R. (2024). Design and evaluation of a global workspace agent embodied in a realistic multimodal environment. Frontiers in Computational Neuroscience, 18:1352685.

Feinberg, T. E. and Mallatt, J. (2020). Phenomenal Consciousness and Emergence: Eliminating the Explanatory Gap. Frontiers in Psychology, 11:1041.

Fulda, F. C. (2020). Biopsychism: Life between computation and cognition. Interdisciplinary Science Reviews, 45(3):315-330.

Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J., Belonax, T., Chen, J., Duvenaud, D., Khan, A., Michael, J., Mindermann, S., Perez, E., Petrini, L., Uesato, J., Kaplan, J., Shlegeris, B., Bowman, S. R., and Hubinger, E. (2024). ALIGNMENT FAKING IN LARGE LANGUAGE MODELS. arXiv preprint.

Koch, C. (2019). The Feeling of Life Itself: Why Consciousness Is Widespread but Can't Be Computed. MIT Press.

Langer, C. and Ay, N. (2021). How Morphological Computation Shapes Integrated Information in Embodied Agents. Frontiers in Psychology, 12:716433.

Libet, B., Gleason, C. A., and Wright, E. W. (1983). TIME OF CONSCIOUS INTENTION TO ACT IN RELATION TO ONSET OF CEREBRAL ACTIVITY (READINESSPOTENTIAL). Brain, 106(3):623-642.

Maoz, U., Yaffe, G., Koch, C., and Mudrik, L. (2019). Neural precursors of decisions that matter-an ERP study of deliberate and arbitrary choice. eLife, 8:e39787.

Marks, S., Treutlein, J., Bricken, T., Lindsey, J., Marcus, J., Mishra-Sharma, S., Ziegler, D., Ameisen, E., Batson, J., Belonax, T., Bowman, S. R., Carter, S., Chen, B., Cunningham, H., Denison, C., Dietz, F., Golechha, S., Khan, A., Kirchner, J., Leike, J., Meek, A., Nishimura-Gasparian, K., Ong, E., Olah, C., Pearce, A., Roger, F., Salle, J., Shih, A., Tong, M., Thomas, D., Rivoire, K., Jermyn, A., MacDiarmid, M., Henighan, T., and Hubinger, E. (2025). AUDITING LANGUAGE MODELS FOR HIDDEN OBJECTIVES. arXiv preprint.

Mashour, G. A., Roelfsema, P., Changeux, J.-P., and Dehaene, S. (2020). Conscious Processing and the Global Neuronal Workspace Hypothesis. Neuron, 105(5):776-798.

Mediano, P. A. M., Rosas, F. E., Farah, J. C., Shanahan, M., Bor, D., and Barrett, A. B. (2022). Integrated information as a common signature of dynamical and information-processing complexity. Chaos: An Interdisciplinary Journal of Nonlinear Science, 32(1):013115.

Miyanishi, Y. and Mitani, K. (2025). Superficial Consciousness Hypothesis for Autoregressive Transformers. In 1st Workshop on Post-Singularity Symbiosis at AAAI-25, Philadelphia, PA, USA.

Oizumi, M., Albantakis, L., and Tononi, G. (2014). From the Phenomenology to the Mechanisms of Consciousness: Integrated Information Theory 3.0. PLoS Computational Biology, 10(5):e1003588.

Scheurer, J., Balesni, M., and Hobbhahn, M. (2024). LARGE LANGUAGE MODELS CAN STRATEGICALLY DECEIVE THEIR USERS WHEN PUT UNDER PRESSURE. In The Twelfth International Conference on Learning Representations (ICLR 2024), Vienna, Austria.

Soon, C. S., Brass, M., Heinze, H.-J., and Haynes, J.-D. (2008). Unconscious determinants of free decisions in the human brain. Nature Neuroscience, 11(5):543-545.

Stefanelli, R. (2023). Theories of consciousness and psychiatric disorders - A comparative analysis. Neuroscience \& Biobehavioral Reviews, 152:105204.

Tononi, G. (2004). An information integration theory of consciousness. BMC Neuroscience, 5(1):42.

Walker, S. I. and Davies, P. C. W. (2013). The algorithmic origins of life. Journal of The Royal Society Interface, 10(79):20120869.

Walker, S. I., Packard, N., and Cody, G. D. (2017). Reconceptualizing the origins of life. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 375(2109):20160337.

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., and Zhou, D. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS'22), pages 2482424837, Red Hook, NY, USA.

# Hummingbird Kernel: Chaotic Lotka-Volterra Sampling for Bayesian Optimization in High-Dimensional Deceptive Landscapes 

Takahito Murakami ${ }^{1}$, Maya Grace Torii ${ }^{1}$, and Yoichi Ochiair ${ }^{1,2}$<br>${ }^{1}$ University of Tsukuba, Japan<br>${ }^{2}$ Pixie Dust Technologies, Inc., Japan takahito@digitalnature.slis.tsukuba.ac.jp


#### Abstract

High-dimensional, multi-modal optimization problems in creative AI remain challenging because many methods converge prematurely. We present the Hummingbird Kernel, a black-box optimization (BBO) framework that integrates a four-species competitive Generalized Lotka-Volterra (gLV) generator into a Bayesian Optimization (BO) loop. The system's deterministic chaotic dynamics yield a structured, nonrepeating sequence of candidate points that may help avoid local optima while maintaining diversity. A deterministic mapping from the gLV state to the target search space provides a controllable exploration mechanism via a single scale parameter that expands or contracts the attractor. In preliminary experiments, Hummingbird achieved lower regret than CMA-ES on deceptive benchmark functions at higher dimensionalities ( $D \geq 50$ ) under our settings. We also provide a small procedural content generation example, suggesting potential suitability for human-in-the-loop workflows where both broad exploration and local refinement are desirable. Although chaotic dynamics are seldom used for candidate generation due to sensitivity to initial conditions and limited controllability, here chaos is confined to a bounded attractor, exposed through a tunable scale, and coupled with BO so that deterministic trajectories act as a reproducible, adjustable proposal process.


## Submission type: Late Breaking Abstracts

Data/Code available at: https://anonymous. 4open.science/r/hummingbird-kernel

## Introduction

Exploring large parameter spaces is central to computational creativity, from procedural level generation to automated design. These tasks are often high-dimensional black-box optimizations whose landscapes can be deceptive, with many poor local optima. CMA-ES often converges to a single basin in deceptive settings, and BO loses surrogate fidelity in high dimensions, leading to premature convergence Frazier (2018); Bellman and Lee (1984).

We propose a complementary strategy that emphasizes persistent and structured exploration. This is particularly important in creative AI, where exploration quality shapes novelty. Building on Lotka-Volterra models Lotka (1925);

Volterra (1926) and the observation that competitive gLV systems with four or more species can exhibit deterministic chaos Vano et al. (2006); Hirsch et al. (2013), we couple a chaotic sampler with BO with the aim of reducing early convergence.

Our contributions were (i) A description of the Hummingbird Kernel: a gLV-driven candidate generator integrated with BO. (ii) Initial comparisons with established optimizers indicating that, in high-dimensional deceptive settings, our method achieved lower regret under our implementation and hyperparameters. (iii) A small procedural content generation case suggesting that controllable exploration may be useful in human-in-the-loop (HITL) workflowsMackay (1990); Amershi et al. (2014); O'Donovan et al. (2015).

## The Hummingbird Kernel

## Generative Core: Four-Species gLV

We use a competitive gLV model:

$$
\frac{d x_{i}}{d t}=x_{i}\left(r_{i}+\sum_{j=1}^{4} A_{i j} x_{j}\right), \quad i=1, \ldots, 4,
$$

where $x_{i}$ is the population of species $i, r_{i}$ are intrinsic growth rates, and $A$ is the interaction matrix. Following Vano et al. (2006), we adopt parameters that yield a robust chaotic attractor. The trajectory $x(t) \in \mathbb{R}^{4}$ is deterministic yet non-repeating within the attractor. A deterministic mapping $\Psi: \mathbb{R}^{4} \rightarrow \mathbb{R}^{D}$ (e.g., bounded affine projection) provides candidate points in the target space, encouraging exploration without collapsing to a single region.

## Controllable Exploration (HITL)

Users modulate exploration by scaling off-diagonal competition terms in $A$ with a scalar $\alpha>0$ such as standard and bellow two modes:

- Divergent - Bold $(\alpha>1)$ : expands the attractor; widerranging search for ideation.
- Refining - Cautious $(\alpha<1)$ : contracts the attractor; concentrated search near incumbents.

This single control aligns with divergent/convergent phases often observed in design workflows O'Donovan et al. (2015).

## Integration with BO

At iteration $t$ : (1) advance the gLV state and form a batch $\left\{x_{t}^{(k)}\right\}$ via $\Psi$; (2) rank candidates using a BO acquisition (e.g., expected improvement) from a GP surrogate Frazier (2018); (3) evaluate selected candidates, update the surrogate, and repeat. The chaotic generator supplies coverage; the surrogate focuses evaluations.

## Why chaos is rarely used, and how it is made usable here

Chaos is usually avoided due to sensitivity, uncontrollable coverage, and drift Hirsch et al. (2013). We address this by using a compact attractor Vano et al. (2006), deterministic integration, a single exploration scale $\alpha$, bounded mapping $\Psi$, and BO filtering..

## Empirical Evaluation

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-055.jpg?height=668&width=718&top_left_y=1176&top_left_x=210)
Figure 1: (A) Regret: best-so-far objective gap and (B) Exploration diversity: Values reflect breadth of search coverage.

## Benchmarks

We evaluated 30 trials of 100 iterations each on three multimodal functions (Ackley, Rastrigin, Levy). Baselines included Random Search, SciPy-LBFGS, BO-LBFGS, BORS, and CMA-ES, whitch is state-of-the art for BBO .

## Results

Figure 1 shows regret and exploration diversity results. Wilcoxon rank-sum tests $(\alpha=0.05)$ showed that HBStandard (gLV → BO) achieved significantly lower final regret than most baselines (e.g., $67.6 \pm 168.4$ vs $212.6 \pm 407.9$,

Table 1: Final regret (mean ± SD). Best per row in bold.
| Fn(D) | CMA-ES | HB-Std. | HB-Bold |
| :--- | ---: | ---: | ---: |
| Ackley 50 | $\mathbf{0 . 8 2 ( 0 . 1 5 )}$ | $3.34(0.42)$ | $4.08(0.50)$ |
| Ackley 100 | $\mathbf{2 . 6 2 ( 0 . 2 7 )}$ | $4.48(0.61)$ | $4.55(0.58)$ |
| Ras. 50 | $438.8(25.4)$ | $\mathbf{2 0 8 . 5 ( 1 9 . 3 )}$ | $258.9(22.1)$ |
| Ras. 100 | $1023.2(41.8)$ | $593.6(35.7)$ | $\mathbf{5 3 2 . 8 ( 2 9 . 9 )}$ |
| Levy 50 | $2.71(0.33)$ | $\mathbf{1 . 3 3 ( 0 . 2 4 )}$ | $3.24(0.29)$ |
| Levy 100 | $11.52(0.98)$ | $7.61(0.71)$ | $\mathbf{6 . 8 8 ( 0 . 6 3 )}$ |


![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-055.jpg?height=398&width=843&top_left_y=714&top_left_x=1073)
Figure 2: Procedural content generation (HITL): (A) Super Mario-like game level design with three exploration parameter controls, and (B) cat blanket pattern design generation.

$p<0.001$ ). HB-Bold performed comparably ( $67.6 \pm 146.6$, $p<0.001$ ), while HB-Cautious was worse. CMA-ES reached targets faster but retained higher regret ( $114.4 \pm$ 263.4, $p<0.001$ ).

For exploration diversity, HB-Standard ( $8.6 \pm 27.1$ ) was higher than CMA-ES ( $6.8 \pm 8.3$, n.s.), while Bold ( $5.9 \pm$ 20.0) and Cautious ( $2.7 \pm 7.5$ ) reduced coverage, consistent with smaller $\alpha$.

## Discussion and Future Work

In deceptive high-dimensional problems, the kernel improves performance by sustaining exploration through bounded chaotic proposals while BO guides evaluations. In a human-in-the-loop platformer task, $\alpha>1$ enabled divergent discovery and $\alpha<1$ supported convergent refinement O'Donovan et al. (2015). Future work will examine larger-scale benchmarks, user-adaptive control of $\alpha$, and broader mappings. Beyond games, we envision applications in layout design, pattern generation, and other domains where controllable exploration is desirable.

## Acknowledgements

This work was supported byJSPS KAKENHI Grant Number 25 KJ 0654 , and used computational resources of supercomputer Fugaku provided by the RIKEN Center for Computational Science through the HPCI System Research project (Project ID:hp250396).

## References

Amershi, S., Cakmak, M., Knox, W. B., and Kulesza, T. (2014). Power to the people: The role of humans in interactive machine learning. AI magazine, 35(4):105-120.

Bellman, R. and Lee, E. (1984). History and development of dynamic programming. IEEE Control Systems Magazine, 4(4):24-28.

Frazier, P. I. (2018). A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811.

Hirsch, M. W., Smale, S., and Devaney, R. L. (2013). Differential equations, dynamical systems, and an introduction to chaos. Academic press.

Lotka, A. J. (1925). Elements of physical biology. Williams \& Wilkins.

Mackay, W. E. (1990). Users and customizable software: A coadaptive phenomenon. PhD thesis, Massachusetts Institute of Technology.

O'Donovan, P., Agarwala, A., and Hertzmann, A. (2015). Designscape: Design with interactive layout suggestions. In Proceedings of the 33rd annual ACM conference on human factors in computing systems, pages 1221-1224.

Vano, J., Wildenberg, J., Anderson, M., Noel, J., and Sprott, J. (2006). Chaos in low-dimensional lotka-volterra models of competition. Nonlinearity, 19(10):2391.

Volterra, V. (1926). Fluctuations in the abundance of a species considered mathematically. Nature, 118(2972):558-560.

# Spreading Online Social Support in an Avatar Communication App 

Masanori Takano ${ }^{1,3}$, and Kenji Yokotani ${ }^{2}$<br>${ }^{1}$ CyberAgent, Inc., Japan<br>${ }^{2}$ Tokushima University, Japan<br>${ }^{3}$ Keio University, Japan<br>m_takano@keio.jp


#### Abstract

Social support improves mental health and well-being and can propagate through social networks as people who receive support are more likely to provide it to others. Yet mapping large-scale offline networks is difficult, which limits empirical analyses of support diffusion. This study examines the diffusion of perceived online social support in a Japanese avatar-based communication app, Pigg Party. We first predict users' perceived support from in-app behavioral logs using machine learning, then use the predicted scores to evaluate diffusion patterns across the social network. A Random Forest model achieves the best out-of-sample accuracy (test correlation $r=0.365$ ), comparable to prior work predicting psychological constructs from behavior. Network analyses show that the effect of a source user's support on others decays with hop distance in a heavy-tailed (approximately powerlaw) manner and remains statistically detectable up to around six hops, suggesting potential mutual reinforcement mechanisms in online communities.


## Submission type: Late Breaking Abstracts

## Introduction

Many previous studies showed that social support benefits mental health and related outcomes (Cohen and Wills, 1985; Rothon et al., 2011). People who receive support tend to provide support in turn, allowing supportive behaviors and perceptions to spread through social ties (Lakon et al., 2017). Understanding the spread pattern of social support is, therefore, central to designing interventions that enhance wellbeing at scale.

However, collecting comprehensive, large-scale offline network data is challenging. In contrast, online platforms provide rich, time-stamped traces of social interaction at scale. Prior studies indicate that online social support is associated with better mental health and well-being (Choudhury and De, 2014; Cole et al., 2017). Particularly, avatar communication apps, which enable quasi-real-time verbal and non-verbal exchanges through virtual bodies in shared spaces, may reproduce key aspects of offline interaction (Takano and Tsunoda, 2019; Takano and Yokotani,
2022). This helps us to study the spread pattern of social support on social networks.

We analyze perceived online social support in Pigg Party, a large Japanese avatar communication app. We (i) build a predictive model of perceived social support from behavioral logs and (ii) analyze the spread pattern of social support on the in-app social network.

## Study 1: Prediction of Perceived Online Social Support

## Dataset and Methods

Participants We conducted two online surveys of Pigg Party players in 2020 (Wave 1: April 26-May 2; Wave 2: June 1-7). Participants consented to academic use of their data. Recruitment occurred within the app. In Wave 1, all players received an in-app announcement; respondents were compensated with virtual coins equivalent to 100 JPY . In Wave 2, Wave 1 participants received an invitation for follow-up with the same incentive. Each participant entered their Pigg Party ID in the questionnaire, allowing linkage of survey responses with behavioral logs and social network data.

We analyzed users who fully completed the survey (Wave 1: $N=2,923$; Wave 2: $N=745$ ). We excluded the fastest $3 \%$ of respondents (completion $<4.10$ minutes) as likely inattentive. We further excluded users whose total app use in the prior month was below the 2.5th percentile of the survey sample, due to insufficient behavioral data.

Target Variable: Perceived Online Social Support Perceived emotional and instrumental support from online friends was measured using validated Japanese scales (Fukuoka and Hashimoto, 1997). Confirmatory factor analyses (maximum likelihood) indicated high internal consistencies: Cronbach's $\alpha$ for emotional and instrumental support were 0.947 and 0.964 (Wave 1) and 0.952 and 0.966 (Wave 2), respectively. As commonly observed (Semmer et al., 2008; Shakespeare-Finch and Obst, 2011), the two support types were strongly correlated. To avoid multicollinearity, we applied principal component analysis (PCA)
to the two subscales and used the first principal component (overall perceived support strength) as the outcome variable according to a previous study (Takano and Yokotani, 2022).

Features From linked behavioral logs before the survey start dates (Wave 1: April 26, 2020; Wave 2: May 31, 2020), we constructed the following features: Number of mutual followers (reciprocal friends) / Counts of various in-app actions (up to the day before survey start) / Dominant usage time window over the preceding month / Avatar's age and gender (not necessarily the user's real attributes) / Total Pigg Party usage time over the preceding month / Avatar outfit/accessory embeddings ( 10 -dim vectors via Paragraph2Vec) / Avatar face-part embeddings ( 10 -dim vectors via Paragraph2Vec) / Total count of owned avatar outfits/accessories / Size of the user's room and count of interior items / Social network degree (edge count) and average edge weight (mean interaction strength) / Wave (1 or 2).

## Model Selection

To predict perceived online social support from behavioral and profile features, we compared four supervised learning algorithms: Random Forest, Elastic Net regression, a linear-kernel Support Vector Machine (SVM), and LightGBM. The full dataset ( $N=3,668$ ) was randomly split into training (80\%), validation (10\%), and test (10\%) partitions. For each algorithm, we performed a grid search over hyperparameters using only the training data to fit candidate models and selected the configuration that minimized the mean absolute error (MAE) on the validation set. The selected model was then evaluated once on the held-out test set to obtain out-of-sample performance.

## Results

Random Forest yielded the best accuracy across all metrics (MAE, RMSE, $R^{2}$, and prediction-observation correlation). Its correlation $r=0.365$ corresponds to a medium effect size in Cohen's criteria and aligns with prior behavior-based prediction of psychological constructs ( $r \approx 0.32-0.45$ ) (Stachl et al., 2020).

## Study 2: Social Network Structure and Perceived Online Social Support

## Dataset and Methods

Predicting Support for Nonparticipants Using Random Forest, we predicted perceived online social support for active users who did not answer the survey on each wave.

Social Network We constructed a social network using behavioral logs between Wave 1 and 2 (see Takano (2023) for details).

Spreading Online Social Support To evaluate diffusion, we examined whether a source user's perceived support at Wave $1\left(y_{1}\right)$ influenced the perceived support of users located $k$ hops away ${ }^{1}$ at Wave $2\left(y_{k, 2}\right)$, controlling for the target users' own Wave 1 levels ( $y_{k, 1}$ ) and covariates:

$$
y_{k, 2}=\beta_{1} y_{k, 1}+\beta_{2 k} y_{1}+\boldsymbol{\zeta}^{\top} \mathbf{C}+\varepsilon,
$$

where $\mathbf{C}$ includes the target users' avatar age and gender, degree and average edge weight, number of owned outfits/accessories, room size and interior count, and recent usage time. The coefficient $\beta_{2 k}$ captures the net influence from the Wave 1 source at hop distance $k$.

## Results

Figure 1 plots $\beta_{2 k}$ against hop distance. The influence decays as $k$ increases and follows an approximately power-law pattern (a heavier tail) rather than exponential decay.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-059.jpg?height=318&width=323&top_left_y=1062&top_left_x=1329)
Figure 1: Hop distances and coefficient $\beta_{2 k}$

## Discussion

This study demonstrates two main findings in an avatarbased online community. First, perceived online social support can be predicted from behavioral and profile signals with moderate accuracy. The Random Forest model achieved an out-of-sample correlation of $r=0.365$, comparable to prior work predicting psychological measures from digital traces (Stachl et al., 2020). This suggests that avatarmediated behaviors encode meaningful information about users' supportive experiences.

Second, perceived online support indicates spreading through the app's social network. A user's support at Wave 1 positively predicts the support of others multiple hops away at Wave 2, after controlling for the targets' prior levels and covariates. The observed decay with hop distance is heavy-tailed-closer to a power law than to exponential-and remains detectable up to around six hops. The heavier tail may indicate mutual reinforcement mechanisms (e.g., reciprocal encouragement) that sustain influence across longer paths.

These results highlight the potential of avatar communication platforms for promoting online social support.

[^1]
## Acknowledgments

This work was supported by JST, PRESTO Grant Number JPMJPR2367, Japan.

## References

De Choudhury, M. and De, S. (2014). Mental Health Discourse on reddit: Self-Disclosure, Social Support, and Anonymity. In Proceedings of the Eighth International Conference on Weblogs and Social Media.

Cohen, S. and Wills, T. A. (1985). Stress, Social Support, and the Buffering Hypothesis. Psychological Bulletin, 98(2):310-357.

Cole, D. A., Nick, E. A., Zelkowitz, R. L., Roeder, K. M., and Spinelli, T. (2017). Online social support for young people: Does it recapitulate in-person social support; can it help? Computers in Human Behavior, 68:456464.

Fukuoka, Y. and Hashimoto, T. (1997). Stress-buffering effects of perceived social supports from family members and friends: A comparison of college students and middle-aged adults. The Japanese Journal of Psychology, 68(5):403-409.

Lakon, C. M., Wang, C., Butts, C. T., Jose, R., and Hipp, J. R. (2017). Cascades of emotional support in friendship networks and adolescent smoking. PLOS ONE, 12(6):e0180204.

Rothon, C., Head, J., Klineberg, E., and Stansfeld, S. (2011). Can social support protect bullied adolescents from adverse outcomes? A prospective study on the effects of bullying on the educational achievement and mental health of adolescents at secondary schools in East London. Journal of Adolescence, 34(3):579-588.

Semmer, N. K., Elfering, A., Jacobshagen, N., Perrot, T., Beehr, T. A., and Boos, N. (2008). The Emotional Meaning of Instrumental Social Support. International Journal of Stress Management, 15(3):235-251.

Shakespeare-Finch, J. and Obst, P. L. (2011). The development of the 2-way social support scale: A measure of giving and receiving emotional and instrumental support. Journal of Personality Assessment, 93(5):483490.

Stachl, C., Au, Q., Schoedel, R., Gosling, S. D., Harari, G. M., Buschek, D., Völkel, S. T., Schuwerk, T., Oldemeier, M., Ullmann, T., Hussmann, H., Bischl, B., and Bühner, M. (2020). Predicting personality from patterns of behavior collected with smartphones. Proceedings of the National Academy of Sciences of the United States of America, 117(30):17680-17687.

Takano, M. (2023). Synchronization of online social rhythms via avatar communications. Journal of Physics: Complexity, 4(1):015010.

Takano, M. and Tsunoda, T. (2019). Self-Disclosure of Bullying Experiences and Social Support in Avatar Communication: Analysis of Verbal and Nonverbal Communications. Proceedings of the International AAAI Conference on Web and Social Media, 13(01):473-481.

Takano, M. and Yokotani, K. (2022). Online Social Support via Avatar Communication Buffers Harmful Effects of Offline Bullying Victimization. Proceedings of the International AAAI Conference on Web and Social Media, 16:980-992.

# A Predictive-Coding-Inspired Variational RNN Model of Focus-Mind-Wandering Transitions under the Free Energy Principle 

Henrique Oyama ${ }^{1}$ and Jun Tani ${ }^{1}$<br>${ }^{1}$ Okinawa Institute of Science and Technology, Japan henrique.oyama@oist.jp


#### Abstract

Mind-wandering (MW) and focus state (FS) are core modes of human cognition, with autonomous transitions between them enabling flexible behavior in dynamic environments. Our recent work introduced a predictive-coding-inspired variational recurrent neural network (PV-RNN) with online adaptation of a meta-prior parameter $w$, balancing top-down predictions and bottom-up sensory inputs. We showed that $w$ can be modulated based on average prediction error, producing autonomous FS-MW transitions without external triggers. We now propose extending this framework with a meta-layer that forecasts future prediction errors. We speculate that conscious awareness of mind-wandering may arise when the predicted error exceeds a certain threshold, enabling proactive modulation of $w$ to redirect attention back to FS.


## Submission type: Late Breaking Abstracts

## Introduction

Mind-wandering involves a dynamic shift from external task focus to internally generated thought streams (Smallwood and Schooler, 2015; Christoff et al., 2016; Seli et al., 2016, 2018; Peral-Fuster et al., 2023). While psychological models have described such alternations, computational mechanisms enabling autonomous FS-MW transitions remain underexplored ((Zukosky and Wang, 2021; SandvedSmith et al., 2021; Idei et al., 2024; Oyama et al., 2025)). Grounded in the Free Energy Principle ((Friston, 2005)), our predictive-coding-inspired recurrent neural network (PVRNN) framework implements a generative model that predicts sensory inputs and infers latent states by minimizing free energy. The meta-prior $w$ regulates the trade-off between the complexity and accuracy terms in free energy, shaping the balance between internally driven imagery and sensory-driven perception ((Ohata and Tani, 2020; Chame et al., 2020; Wirkuttis et al., 2023)).

## Model and Methods

The base PV-RNN (Fig. 1) consists of hierarchical deterministic and stochastic latent variables, supporting multitimescale sequence learning. Given a PV-RNN with $L$ lay-
ers, predicting a $T$ time series sensory inputs, the free energy $\mathcal{F}$ can be written as (Ohata and Tani, 2020):

$$
\mathcal{F}=\underbrace{\sum_{t=1}^{T}\left[\sum_{l}^{L} \frac{\mathbf{w}^{l}}{R_{z}^{l}} \delta(l, r, t)\right]}_{\text {complexity }}-\underbrace{\frac{1}{R_{X}}\left[\sum_{t=1}^{T}\left\|\mathbf{X}_{t}-\overline{\mathbf{X}}_{t}\right\|_{2}^{2}\right]}_{\text {accuracy }}
$$

where $\mathbf{w}^{l}$ is meta-prior specific to $l$-th layer, $\overline{\mathbf{X}}$ denotes the prediction output of the PV-RNN, $R_{X}$ is the data dimension, $R_{z}^{l}$ is the number of $\mathbf{z}$ variables in each layer, $\delta(l, r, t)$ is computed as in (Oyama et al., 2025).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-062.jpg?height=796&width=796&top_left_y=1356&top_left_x=1098)
Figure 1: Internal structure of base PV-RNN under metalayer and $w$ modulation. Solid blue lines represent the generative process, while dotted red lines indicate the inference process. The shaded area shows an inference window of length 3.

During training, $w$ is fixed; during inference, $w$ adapts probabilistically according to the average prediction error
over a sliding window. In particular, if the error decreases near a minimal threshold, $w$ switches to a higher value (top-down driven prediction dominates, resembling MW), whereas if the error increases near a maximal threshold, $w$ switches to a lower value (bottom-up processing dominates, resembling FS). State transitions follow a probabilistic switching rule, enabling variability in dwell times and stochasticity in switching.

## Results

Training involved probabilistic switching among four cyclic patterns, whereas inference used a single trained pattern as the target sequence. The model learned their temporal structure and generated autonomous FS-MW transitions during inference: (A) FS mode (low $\boldsymbol{w}$ ): results in accurate sensory reconstruction and lower prediction error with larger Kullback-Leibler divergence between prior and posterior (Fig. 2). (B) MW mode (high $\boldsymbol{w}$ ): results in divergence from sensory inputs and higher prediction error (Fig. 3).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-063.jpg?height=789&width=796&top_left_y=1114&top_left_x=175)
Figure 2: From top to bottom: inference output trajectory with low $w$ value and average prediction errors over the inference window at time step 100. Inference $-X$ and Inference $-Y$ correspond to the first and second dimensions of the inference output trajectory, respectively.

These dynamics reproduce qualitative features of autonomous transitions between FS and MW, including variability in task-aligned behavior, time-on-task drift toward MW, and a nonlinear relationship with task difficulty.

## Planned Meta-Layer Extension

We propose a meta-layer, implemented as a variational RNN, on top of the base PV-RNN (Fig. 1) to forecast future

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-063.jpg?height=729&width=782&top_left_y=274&top_left_x=1103)
Figure 3: From top to bottom: inference output trajectory with high $w$ value and average prediction errors over the inference window at time step 290. Inference $-X$ and Inference $-Y$ correspond to the first and second dimensions of the inference output trajectory, respectively.

prediction errors over a short horizon. Conscious awareness of MW may arise when the predicted future error exceeds a threshold, signaling a potential mismatch between anticipated and actual sensory alignment. This event could trigger a shift back to FS by adjusting $w$ preemptively. The meta-layer will be trained jointly with the base PV-RNN and used during inference as a meta-cognitive mechanism that explains how an agent may become consciously aware of MW.

## Conclusion

This work offers a Free Energy Principle account of spontaneous cognitive state transitions, implemented in a PV-RNN model that autonomously alternates between FS and MW. Extending the model with a meta-layer that forecasts future prediction errors provides a principled computational account of how conscious awareness of mind-wandering may emerge, offering a framework to explore the interplay between prediction, awareness, and cognitive state dynamics in both natural and artificial agents.

## Acknowledgements

This work was supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI Early-Career Scientists Grant [25K21307] and JSPS KAKENHI Transformative Research Area (A): unified theory of prediction and action [24H02175].

## References

Chame, H. F., Ahmadi, A., and Tani, J. (2020). A hybrid humanneurorobotics approach to primary intersubjectivity via active inference. Frontiers in Psychology, 11:3207.

Christoff, K., Irving, Z. C., Fox, K. C., Spreng, R. N., and Andrews-Hanna, J. R. (2016). Mind-wandering as spontaneous thought: a dynamic framework. Nature reviews neuroscience, 17(11):718-731.

Friston, K. (2005). A theory of cortical responses. Philosophical transactions of the Royal Society B: Biological sciences, 360(1456):815-836.

Idei, H., Suzuki, K., and Yamashita, Y. (2024). Awareness of being: A computational neurophenomenological model of mindfulness, mind-wandering, and meta-attentional control.

Ohata, W. and Tani, J. (2020). Investigation of the sense of agency in social cognition, based on frameworks of predictive coding and active inference: A simulation study on multimodal imitative interaction. Frontiers in Neurorobotics, 14:61.

Oyama, H., Matsumoto, T., and Tani, J. (2025). Modeling autonomous shifts between focus state and mind-wandering using a predictive-coding-inspired variational recurrent neural network. Frontiers in Computational Neuroscience, Volume 19-2025.

Peral-Fuster, C. I., Herold, R. S., Alder, O. J., Elkelani, O., RibeiroAli, S. I., Deane, E. M., Martindale, A. P., Qi, Z., Westling, C. E., and Witchel, H. J. (2023). Intentional mind wandering is objectively linked to low effort and tasks with high predictability. In Proceedings of the European Conference on Cognitive Ergonomics 2023, pages 1-8.

Sandved-Smith, L., Hesp, C., Mattout, J., Friston, K., Lutz, A., and Ramstead, M. J. (2021). Towards a computational phenomenology of mental action: modelling meta-awareness and attentional control with deep parametric active inference. Neuroscience of consciousness, 2021(1):niab018.

Seli, P., Konishi, M., Risko, E. F., and Smilek, D. (2018). The role of task difficulty in theoretical accounts of mind wandering. Consciousness and Cognition, 65:255-262.

Seli, P., Risko, E. F., Smilek, D., and Schacter, D. L. (2016). Mindwandering with and without intention. Trends in cognitive sciences, 20(8):605-617.

Smallwood, J. and Schooler, J. W. (2015). The science of mind wandering: Empirically navigating the stream of consciousness. Annual review of psychology, 66(1):487-518.

Wirkuttis, N., Ohata, W., and Tani, J. (2023). Turn-taking mechanisms in imitative interaction: Robotic social interaction based on the free energy principle. Entropy, 25(2):263.

Zukosky, M. and Wang, R. F. (2021). Spontaneous state alternations in the time course of mind wandering. Cognition, 212:104689.

# Investigating dynamics of Neural Cellular Automata in complex systems 

Vaibhav Mahajan ${ }^{1}$, and Soumya Banerjee ${ }^{1}$<br>${ }^{1}$ University of Cambridge, United Kingdom<br>sb2333@cam.ac.uk


#### Abstract

Neural cellular automata (NCA) provide a powerful computational paradigm for modelling morphogenetic processes through local interactions and self-organization. We apply NCAs to a number of prototypical complex systems ranging from morphogenesis to reaction-diffusion systems. We explore the capacity of NCA to not only replicate complex visual patterns, but also to learn the underlying update rules of dynamic systems from spatiotemporal snapshots. We reproduce the behaviour of a morphogenesis system through various training regimes and demonstrate how training strategies critically influence the ability of the NCA to grow, persist, and regenerate patterns. We find that NCAs cannot be applied "out of the box" to these diverse problems but must be adapted. We introduce a stratified multi-step training process that can be used to train NCAs to replicate diverse complex systems. Lastly we find that NCAs use the hidden channels to generalize to novel behaviour. We further analyse the role of hidden channels in encoding spatial memory and guiding complex pattern formation. Our experiments provide new insights into how neural CA can be adapted as generalpurpose models for learning, replicating, and possibly innovating system dynamics. Our findings illustrate the versatility of NCA as a self-organising and rule-learning system (albeit with complex training regimes) and suggest broader applications in modelling natural and artificial systems.


## Submission type: Late Breaking Abstract

## Introduction

Cellular automata have long served as a compelling framework for studying complex systems through simple local rules. Recent work by Mordvintsev et al. (2020) introduced a neural formulation of cellular automata (NCA) that combines traditional automata with neural networks. This approach encodes the state of each cell in multiple channels and evolves the system through convolutional update rules. The original NCA model demonstrated an impressive capability to self-organise and regenerate patterns. However, it remains an open question whether such systems can generalise and be used to infer rules underlying arbitrary spatiotemporal phenomena.

In this paper, we explore the potential of NCA to learn the dynamics of a system from observation alone. We study how carefully curated training regimes can induce desired behaviours such as long-term stability and structural recovery. We further analyse the role of hidden channels in encoding spatial memory and guiding complex pattern formation. Our experiments provide new insights into how neural CA can be adapted as general-purpose models for learning, replicating, and possibly innovating system dynamics.

## Inferring the Rules

We investigate whether neural cellular automata can be used to reproduce complex behaviour by inferring the rules of a system given only snapshots of how the system evolves. The first training regime used was to simply provide the model with a snapshot given as input and its successor as the target, so that the model would learn the update rule. However, we found that although the model was quite accurate when predicting a single step, it failed to learn the long-term dynamics of the system, displaying instability over longer periods.

To allow the model to learn to exhibit long-term dynamics, the loss must be computed over multiple steps. The multi-step training regime randomly chooses a number $n$ and computes the loss between the model prediction of the state $n$ steps into the future and the true state of the frame at that time step. This training regime is much more effective than the single-step regime, exhibiting the correct growth and stability behaviour; however, the model still suffered from the fact that the growth stage in the training data was relatively small, which meant that the model was not able to reproduce the pattern to the level of detail of the original.

The behaviour of the morphogenesis system can be broadly categorised into two parts: the growing stage, where the seed grows into the pattern over successive updates, and the stable stage, where once the target pattern has been created, successive updates do not change the pattern. The state of the system is controlled by the hidden channels. A middle ground is when the system is regenerating a pattern: here, the system maintains the persistence behaviour for most of the image, but exhibits growth on the damaged section. In
order to ensure that the NCA we trained also exhibited this behaviour, multiple runs of the model were used as training data: one where the system is growing, one where the system is stable, and one where the system is regenerating. Stratified sampling from each of these phases was used per batch, with an emphasis on the growing phase. This ensures that the model does not just learn the simpler behaviour of persisting an already grown image, but is also able to grow it from scratch like the original morphogenesis system. A similar strategy is effective at teaching the NCA to replicate the behaviour of Conway's Game of Life and the Be-lousov-Zhabotinsky reaction.

## Influence of Hidden Channels

The hidden channels of the model are crucial to controlling the behaviour of the morphogenesis system. Although the morphogenesis model is only directly trained by taking the L2 loss over the RGBA channels, the hidden channels govern the behaviour of the model. During the growth phase, all channels are active in the pattern. However, in the stable phase, different channels are active in different physical locations of the pattern, using the channels to encode the final pattern.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-067.jpg?height=259&width=688&top_left_y=1283&top_left_x=242)
Figure 1: Snapshot of the morphogenesis model displaying all 16 channels at the growing (top) and stable (bottom) phases. The first image is the RGB channels, the second image is of the alpha channel which is used to determine whether a cell is considered alive or dead, and the four other images display three of the hidden channels each.

A key finding of this work is insight into how information about the spatial location of cells is encoded in the hidden channels, allowing the different features of the pattern to be grown at different locations despite using a common update rule for all the cells.

Figure 1 shows a visualisation of the channels for when the system is in the growing and stable phases. When the system is still growing, the expression of all the channels is still changing for each cell, and there is overlap between the locations where each channel takes a high value. However, in the stable state the cells differentiate themselves, using different channels to encode information about their role within the pattern.

To analyse the spatial distribution of cells with similar states, clustering is performed on the cell states. The kmeans clustering algorithm was applied to cluster the cell

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-067.jpg?height=433&width=696&top_left_y=278&top_left_x=1146)
Figure 2: Visualisation of clustering of the cells of the stable state of the system into five clusters.

states, visualised in Figure 2. During the growing stage, the cells are all similar, however as the system grows, the cells differentiate and specialise. The cells which form the boundary all belong to the same cluster, suggesting that the boundary cells encode that information in the hidden channels to regulate the growth of the pattern. This insight into the behaviour of the morphogenesis system helps explain previous results, which show that reducing the number of channels available to an NCA model diminishes its ability to perform the task of image morphing Richardson et al. (2024). Using fewer channels limits the model's capacity to encode information about the pattern features, thereby affecting the model's ability to produce detailed patterns and exhibit complex behaviour.

## Conclusion

Our study reaffirms the ability of neural cellular automata to replicate, persist, and regenerate complex patterns through localised interactions. By designing training strategies that emphasise different phases of morphogenetic evolution, growth, stability, and regeneration, we demonstrate that NCAs can internalise sophisticated rule-based behaviours from data alone. The stratified multi-step training regime proves especially effective in balancing these phases and in preventing models from degenerating into trivial persistence or overfitting to short-term dynamics.

We further highlight the critical role of hidden channels in modulating emergent behaviours, suggesting that these internal representations function analogously to latent fields in biological development. Our experiments show that NCAs not only reconstruct known dynamics but can also generalise to novel behaviours such as pattern reproduction, hinting at their broader applicability in artificial life. As neural CA continue to blur the boundaries between rule-based simulation and learned behaviour, future work should explore their use in more abstract domains, from agent-based modelling to distributed computation, and investigate the extent to which they can discover interpretable representations of the systems they emulate.

## References

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing neural cellular automata. Distill. https://distill.pub/2020/growing-ca.

Richardson, A. D., Antal, T., Blythe, R. A., and Schumacher, L. J. (2024). Learning spatio-temporal patterns with neural cellular automata. PLOS Computational Biology, 20(4):1-27.

# LLM-Driven Social Behaviors of Virtual Doubles in Shared Mixed Reality 

Chris Elvis Leisi, Florian Christoph Bruggisser and Oliver Sahli<br>Immersive Arts Space, Zurich University of the Arts, Switzerland<br>chris.leisi@zhdk.ch


#### Abstract

We present doppelgaenger:apparatus, an ongoing research-through-design project exploring how large language models (LLMs) can control virtual doubles in a mixed reality (MR) environment. Building on previous work in conversational agents, we extend LLM capabilities beyond dialogue to drive social behaviors, gestures, and context-sensitive actions of embodied agents. Visitors encounter an AI-controlled 3D reconstruction of themselves, created from a single image using neural volumetric capture and engage in uncanny, co-present interactions with their doubles. Our system integrates ChatGPT as a decision-making layer, mapping generated output to a structured set of animation and behavioral commands within the game engine. Early prototypes demonstrate how prompt engineering and multimodal input (e.g., user viewpoint, spatial metadata) enable the LLM to enact believable agent responses in real time. This work raises research questions around social presence, co-presence, and the plausibility of interactions with LLM-controlled agents, while also surfacing ethical concerns around bias, representation, and privacy in generative embodied systems.


## Submission type: Late Breaking Abstracts

## Introduction

The rapid increase in size of large language models (LLMs) such as ChatGPT has profoundly impacted how we access information, generate and refine text, and engage in conversational interfaces in research and artistic domains (Telenti et al., 2024; Messner et al., 2025; Franceschelli and Musolesi, 2024). Beyond standard dialogue, LLMs can drive Embodied Conversational Agents (ECAs) to enable dynamic, context-aware communication behaviors (Li et al., 2022). However, most of the existing work emphasizes the generation of dialogue and basic gestures while interacting with such agents (Wan et al., 2024). In another project, using LLMs to create personalities for ECAs in Virtual Reality (VR) (NGO et al., 2018), they underline the importance of well-crafted verbal interactions in shaping user perceptions of such agents. The prevailing research relied primarily on verbal cues; we see the need for more research in the area
of LLMs controlling other behaviors than just chat structured dialog use. The work of Park et al. (2023) shows that LLMs can simulate more complex human behaviors such as cooking breakfast, starting conversations or planning parties. Yoshida et al. (2025) used an LLM that converts natural language instructions into Python code through chain of thought reasoning (Wei et al., 2022), enabling actuator control and movement of a humanoid robot. Based on these projects, we want to engage the following questions with our ongoing research. Can an LLM (R1) simulate social behaviors other than dialog of a virtual agent confronting a real user? (R2) create interesting and believable interactions between a real person and its virtual double? (R3) control agent behave and react to multiple real visitors and other virtual doubles?

## Previous Work and Methods

To answer this question, we extend the artistic experience doppelgaenger:apparatus, a multiuser MR (Milgram and Kishino, 1994) work that lets users confront their computergenerated 3D double in their actual surroundings (Balakrishnan et al., 2021). Before the experience, a frontal 2D image of the user is taken and quickly transformed into an animated 3D reconstruction using Nevo (Neural Volumetric Capture) (Bruggisser et al., 2024). Nevo enables rapid creation of digital humans by combining machine learning models with traditional algorithms to automatically convert a single 2D human image into a volumetric 3D-Mesh. After this, users put on the MR head-mounted display and the experience begins. Through the headset, the real environment captured by its camera is visible, with the digital double overlayed in front of them. Historically, the doppelgaenger was seen as a ghostly counterpart of a living person, interpreted as an omen of death (Freud, 2017). We draw on this to probe how digital doppelgaengers generate uncanny tension between one's physical body and its virtual counterpart. In the context of machine learning, Steyerl's notion of the "poor image" Steyerl et al. (2009) is particularly fitting: a highly compressed, rapidly circulating digital artefact that loses fidelity through each creation step but gains immedi-
acy and psychological impact. Similarly, the uncanny valley (Mori, 1970) describes how affinity for an artificial agent rises with human-likeness until subtle mismatches trigger eeriness and discomfort (Cheetham et al., 2011; RosenthalVon der Pütten et al., 2019). Following Milgram's (1994) reality-virtuality continuum, we employ passthrough MR to integrate the virtual double into the user's physical space, enhancing immediacy and co-location. Presence, the psychological 'sense of being there' (Slater and Wilbur, 1997), is crucial for immersion. Believable characters and environments foster suspension of disbelief, while contingent, credible scenarios deepen engagement and strengthen the plausibility illusion - the sense the virtual interaction is real (Slater 2009, Slater et al. 2022). Co-presence and social presence are also crucial, with co-presence defined as the subjective 'sense of being with another' (Erickson-Davis et al., 2021). We apply an iterative research-through-design approach combining concept development, prototyping, and public showings. Each cycle incorporates user feedback and observation. Evaluation includes quantitative surveys, qualitative behavioral video analysis, and interviews to assess co-presence and shifts in social perception with the LLMcontrolled doppelgaenger.

## Casy Study: LLM-Controlled Virtual Doubles

By adding LLMs to the doubles, we aim to make their behaviors feel more natural. Technically, the system is based on OpenAI's ChatGPT. The LLM acts as the decisionmaker, controlling a complex state manager of prerecorded animations. At the start of the experience, we send an initial prompt that establishes the scene and instructs the LLM to respond not as a conversation partner, but by generating actions and reactions. We define a fixed set of high-level commands (e.g., WalkToTarget(target, speed), LookAtTarget(target), SetScaredLevel(level), PlayAnimation(name)) that the LLM can trigger to control the double within the game engine. Currently, a system analyzes returned text for these commands, but we plan to adopt OpenAI's Tool System to let the LLM directly call these functions.

In this next iteration of doppelgaenger:apparatus, we will explore image analysis to generate more believable agent behaviors. The system will stream live-rendered images from the user's head position, capturing their viewpoint towards the virtual doppelgaenger, alongside spatial metadata (object locations, orientations). Through strategic prompt design, the LLM is presented as being the "real person" itself, and the users are its doppelgangers. The LLM will process this multimodal input to decide the double's next moves, forming a closed-loop perceptual feedback system. Early tests show that response time is critical for fluid interactions, so we use the streaming API to begin receiving characters while the LLM is still processing, reducing perceived latency. Beyond agent behavior, we must design an interactive environment with object-based actions (e.g., sit, hide, ges-
ture). Since we use Passthrough Mixed Reality(MR), this environment must also exist physically, allowing the user and virtual double to interact with the same real objects, enhancing the sense of shared presence in MR.

## Further development

The current implementation utilizes OpenAI's ChatGPT. We plan to evaluate different LLMs based on behavioral output and response time, potentially including local models running directly on the headset, provided they meet performance requirements. The LLM acts as the decision-maker, controlling a complex state manager of prerecorded animations. At the start of the experience, we send an initial prompt that establishes the scene and instructs the LLM to respond not as a conversation partner, but by generating actions and reactions. We define a fixed set of high-level commands (e.g., WalkToTarget(target, speed), LookAtTarget(target), SetScaredLevel(level), PlayAnimation(name)) that the LLM can trigger to control the double within the game engine. Currently, a system analyzes returned text for these commands, but we plan to adopt OpenAI's Tool System to let the LLM directly call these functions. API ${ }^{1}$ shows promise for combining image recognition with large language models; however, it is still experimental and relatively unstable. Future iterations may leverage this capacity to use the captured camera image of the headset itself. Another research avenue concerns balancing pre-scripted elements with real-time generative responses (Park et al., 2023). For example, having visitors complete a brief questionnaire focused on the Big Five personality traits could provide the agent with baseline personality data, making its behaviors more tailored and believable (Soto and Jackson, 2013).

It is important to note that any biases or limitations inherent in large language models will inevitably carry over to their embodied generative agents (Park et al., 2023). Addressing such biases is therefore a crucial objective of future work. One strategy is to diversify the training data for the visual reconstruction, including representations that move beyond binary gender categories, integrate physical disabilities (e.g., wheelchair users or individuals with limb differences), and feature non-normative body proportions. Such diversity can help counteract "normative discursive frameworks" around body types (Niehaus and Fiebrink, 2021) and broaden the inclusivity of embodied virtual agents.

## Acknowledgements

This work was supported by the Swiss National Science Foundation (SNSF) and by the Leading House Asia Programme of SERI.

## References

Balakrishnan, S., Hameed, M. S. S., Venkatesan, K., and Aswin, G. (2021). Interaction of spatial computing in augmented reality.

[^2]In 2021 7th International Conference on Advanced Computing and Communication Systems (ICACCS), volume 1, pages 1900-1904. IEEE.

Bruggisser, F. C., Leisi, C. E., Lund-Jensen, P., Fröhlich, M., and Salter, C. L. (2024). reconfigure: Confronting audiences with digital doppelgängers. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 7(4):1-10.

Cheetham, M., Suter, P., and Jäncke, L. (2011). The human likeness dimension of the "uncanny valley hypothesis": Behavioral and functional mri findings. Frontiers in Human Neuroscience, volume 5-2011.

Erickson-Davis, C., Luhrmann, T. M., Kurina, L. M., Weisman, K., Cornman, N., Corwin, A., and Bailenson, J. (2021). The sense of presence: Lessons from virtual reality. Religion, brain \& behavior, 11(3):335-351.

Franceschelli, G. and Musolesi, M. (2024). On the creativity of large language models. AI \& SOCIETY, pages 1-11.

Freud, S. (2017). The uncanny. In Romantic Writings, pages 318325. Routledge.

Li, J., Tang, T., Zhao, W. X., Nie, J.-Y., and Wen, J.-R. (2022). Pretrained language models for text generation: A survey.

Messner, W., Greene, T., and Matalone, J. (2025). Express: From bytes to biases. investigating the cultural self-perception of large language models. Journal of Public Policy \& Marketing, page 07439156251319788.

Milgram, P. and Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12):1321-1329.

Mori, M. (1970). The uncanny valley: the original essay by masahiro mori. Ieee Spectrum, 6(1):6.

NGO, B., HARMAN, J., and TÜRKAY, S. (2018). Using llms to develop personalities for embodied conversational agents in virtual reality.

Niehaus, K. H. and Fiebrink, R. (2021). Making up 3d bodies: Artistic and serendipitous modeling of digital human figures. Proc. ACM Comput. Graph. Interact. Tech., 4(2).

Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST '23, New York, NY, USA. Association for Computing Machinery.

Rosenthal-Von der Pütten, A. M., Krämer, N. C., Maderwald, S., Brand, M., and Grabenhorst, F. (2019). Neural mechanisms for accepting and rejecting artificial social partners in the uncanny valley. Journal of Neuroscience, 39(33):6555-6570.

Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535):3549-3557.

Slater, M., Banakou, D., Beacco, A., Gallego, J., Macia-Varela, F., and Oliva, R. (2022). A separate reality: An update on place illusion and plausibility in virtual reality. Frontiers in virtual reality, 3:914392.

Slater, M. and Wilbur, S. (1997). A framework for immersive virtual environments (five): Speculations on the role of presence in virtual environments. Presence: Teleoperators \& Virtual Environments, 6(6):603-616.

Soto, C. J. and Jackson, J. J. (2013). Five-factor model of personality. Journal of Research in Personality, 42:1285-1302.

Steyerl, H. et al. (2009). In defense of the poor image. $e$-flux journal, 10(11):1-9.

Telenti, A., Auli, M., Hie, B. L., Maher, C., Saria, S., and Ioannidis, J. P. (2024). Large language models for science and medicine. European journal of clinical investigation, 54(6):e14183.

Wan, H., Zhang, J., Suria, A. A., Yao, B., Wang, D., Coady, Y., and Prpa, M. (2024). Building llm-based ai agents in social virtual reality. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA '24, New York, NY, USA. Association for Computing Machinery.

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837.

Yoshida, T., Masumori, A., and Ikegami, T. (2025). From text to motion: grounding gpt-4 in a humanoid robot "alter3". Frontiers in Robotics and AI, 12:1581110.

# Population of Thoughts: Exploring Open-Ended Evolution and Discovery for Test-time Compute and Reasoning 

Bryan Lim ${ }^{1}$, and Adam Gaier ${ }^{1}$<br>${ }^{1}$ Autodesk Research<br>bryan.lim@autodesk.com, adam.gaier@autodesk.com


#### Abstract

Large language models are increasingly trained for reasoning, but current methods optimize solely for correct answers, producing narrow reasoning patterns. This narrow objective leaves the space of possible reasoning strategies largely unmapped. We apply Quality-Diversity (QD) algorithms to discover diverse reasoning paths, asking not whether models can solve problems, but if we can catalyze models to reason in different ways. We introduce Population of Thoughts (PoTs) , an algorithm that applies an open-ended evolutionary search process with test-time compute resources. Inspired by QD algorithms, PoTs maintains and cultivates a population of diverse reasoning paths. The result is a rich archive of varied, high-quality reasoning strategies - a conceptual map of the model's cognitive landscape for a given problem. Early results show promise that this method could help uncover novel and diverse behaviors, modes and paths of thought.


## Submission type: Late Breaking Abstract

## Introduction

Large language models are shifting from knowledge repositories to reasoning engines. Test-time compute techniques like Chain-of-Thought (CoT) are driving this change by generating explicitly reasoning steps during inference.

Given that capabilities like reasoning can emerge from prompting (Wei et al., 2022; Kojima et al., 2022) or decoding (Wang and Zhou, 2024) strategies, the full space of model capabilities likely remains unexplored. Even on the level of reasoning, there exist many different phases and types of reasoning which could lead to better performance.

We build on this idea and explore an open-ended evolutionary search across diverse reasoning traces to solve problems. Inspired by systems such as technological and cultural evolution, which maintain populations and archives of artifacts throughout the search process, open-ended search in the vast prompt-token-reasoning space could uncover further emergent but hidden capabilities and behaviors of these models. As a first step towards this goal, we contend that a population of diverse thoughts can be evolved through variations in prompts within the context of a single question or query,
applying evolutionary pressure to guide the LLM towards solutions that are both high-quality but also cognitively diverse.

We introduce Population of Thoughts (PoTs), a modular framework that applies Quality-Diversity algorithms to evolve populations of reasoning traces. This approach enables investigation of compelling questions: whether evolutionary search can discover reasoning patterns outside human cognitive habits; whether accuracy and diversity are complementary rather than competing objectives; and whether breakthrough capabilities hide not in model scale but in unexplored regions of token space. This paper sketches the framework, and present initial results, leaving further exploration and validation for future work.

## Method

PoTs is a test-time compute algorithm that can be used with any foundation model. It has the same building blocks and core components as QD algorithms. The primary difference is that variations are applied on the token and prompt level. Additionally, the algorithm cannot use the true answer or a verifier to evaluate the "quality" and "novelty" of the answer and generated reasoning path. These measures must be derived from the properties of the solutions themselves, or proxy evaluators such as model judges, value functions or surrogate models. Given space limitations, a more comprehensive related work section can be found in the Appendix.

The two core components of open-ended evolution with PoTs are (i) maintaining a diverse archive and (ii) variations to generate new solutions.

Archive The population or archive passively acts as a storage and memory so solutions are not pre-maturely discarded for "poor" performance, and can potentially act as stepping stones later on in evolution for better solutions (Gaier et al., 2019). The elements of competition which creates evolutionary pressure for improvement in the archive come from the update criteria, which is to either be (i) a new niche, novel/interesting with respect to the current population, or (ii) better quality than solutions within your niche. In our experiments, solution quality is measured by the number of other population members converging on the same answer, analo-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-075.jpg?height=723&width=1728&top_left_y=269&top_left_x=171)
Figure 1: Left: Illustration of the modular open-ended evolutionary loop of PoTs to grow and discover and increasingly diverse archive of thoughts. Right: Reasoning mode composition across reasoning and language 'sparks'. Each reasoning trace is divided into early, middle, and late phases, with reasoning chunks classified by an LLM into cognitive modes. Percentages represent mode prevalence averaged across all solutions in the archives produced from PoTS run on 10 separate questions.

gous to the self-consistency metric in multi-output prompting, where the consensus among multiple outputs signals a higherconfidence result (Wang et al., 2022).

For diversity, we use the classification coming from the spark words directly as niches. In this case, we can obtain these descriptors directly from the prompt. Any other measures or minimal criteria (Lehman and Stanley, 2010; Brant and Stanley, 2017) could be used as in PoTs, such as embeddings (numerical or semantic), LLM-judges, or interestingness criteria (Zhang et al., 2023) of the resulting reasoning traces and outputs of the model.

Variations The modularity of PoTs allows any combination of variations to be applied to encourage different samples: sampling new prompts/words, to decoding strategies (i.e. top-K, likelihoods of tokens etc.), few-shot prompting using solutions in the archive, evolved solutions or meta prompts, multiagent debate (Du et al., 2023) and more. This maximises and is complementary to all the various strategies from prior and future work done in prompting and decoding. In our experiments, we rely on simple seed word prompting to initiate varied and different reasoning. We augment the standard prompt with a "spark word." For example: [Question] Solve the problem via the following strategy: "analogies" or [Question] Think and reason using the following language, characters or script: "Swahili".

Initialized from a set of sampled traces, PoTs then follows an evolutionary loop of selection, mutation, evaluation, and population update. After the test-time budget is exhausted, the result is a population of diverse thought traces. If required, a final answer from PoTs can be determined by a majority
vote over the population.

## Early Results and Discussion

As an initial experiment, we evaluate on the GSM8k (Cobbe et al., 2021) dataset. Though solving these baseline questions is trivial for modern LLMs, our interest lies in the ability of PoTs to catalyze a diversity of reasoning paths to the answer. To evaluate the path of each response, we chunk the reasoning traces by paragraphs (i.e. $\backslash \mathrm{n} \backslash \mathrm{n}$ ) and use an LLM-judge to classify the chunks in to the categories.

With this setup, we observe that targeted prompt modifications can systematically alter reasoning patterns to favor distinct cognitive paths, enabling exploration within the space of reasoning modes. Figure 1 reveals that different experimental conditions shift the distribution of cognitive operations across temporal phases. Sparks show distinct signatures: "elimination" increases problem setup and information extraction, while "make a table" elevates interpretation and restating. Language variants also display characteristic patterns: Greek amplifies reflection, Chinese boosts interpretation. These shifts demonstrate PoTs' ability to navigate cognitive reasoning space with controllable strategy distributions.

Reasoning is search, but current methods optimize narrowly for correctness. Our results reveal reasoning traces are malleable, simple variations produce different cognitive patterns, evidence that diversity exists but remains unexplored. Just as QD reframed evolutionary search by valuing how solutions behave, not just whether they succeed, divergent search principles can unlock a richer landscape than convergent training will ever discover.

## References

Brant, J. C. and Stanley, K. O. (2017). Minimal criterion coevolution: a new approach to open-ended search. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 67-74.

Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. (2021). Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.

Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. (2023). Improving factuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning.

Gaier, A., Asteroth, A., and Mouret, J.-B. (2019). Are quality diversity algorithms better at generating stepping stones than objective-based search? In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 115116.

Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. (2022). Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-22213.

Lehman, J. and Stanley, K. O. (2010). Revising the evolutionary computation abstraction: minimal criteria novelty search. In Proceedings of the 12th annual conference on Genetic and evolutionary computation, pages 103-110.

Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. (2022). Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.

Wang, X. and Zhou, D. (2024). Chain-of-thought reasoning without prompting. Advances in Neural Information Processing Systems, 37:66383-66409.

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837.

Zhang, J., Lehman, J., Stanley, K., and Clune, J. (2023). Omni: Open-endedness via models of human notions of interestingness. arXiv preprint arXiv:2306.01711.

# Neural Cellular Automata: From Cells to Pixels 

Ehsan Pajouheshgar ${ }^{1}$, Yitao Xu ${ }^{1}$, Ali Abbasi ${ }^{2 \dagger}$, Alexander Mordvintsev ${ }^{3}$, Wenzel Jakob ${ }^{1}$, Sabine Süsstrunk ${ }^{1}$<br>${ }^{1}$ School of Computer and Communication Sciences, EPFL, Switzerland ${ }^{\dagger}$ Work done during internship at EPFL<br>${ }^{2}$ Sharif University of Technology, Iran; ${ }^{3}$ Google Research, Zurich, Switzerland<br>\{ ehsan.pajouheshgar, yitao.xu, sabine.susstrunk wenzel.jakob \}@epfl.ch<br>a80.abbasi@gmail.com moralex@google.com


#### Abstract

Neural Cellular Automata (NCAs) are bio-inspired systems in which identical cells self-organize to form complex and coherent patterns by repeatedly applying simple local rules. NCAs display striking emergent behaviors including selfregeneration, generalization and robustness to unseen situations, and spontaneous motion. Despite their success in texture synthesis and morphogenesis, NCAs remain largely confined to low-resolution grids. This limitation stems from (1) training time and memory requirements that grow quadratically with grid size, (2) the strictly local propagation of information which impedes long-range cell communication, and (3) the heavy compute demands of real-time inference at high resolution. In this work, we overcome this limitation by pairing NCA with a tiny, shared implicit decoder, inspired by recent advances in implicit neural representations. Following NCA evolution on a coarse grid, a lightweight decoder renders output images at arbitrary resolution. We also propose novel loss functions for both morphogenesis and texture synthesis tasks, specifically tailored for high-resolution output with minimal memory and computation overhead. Combining our proposed architecture and loss functions brings substantial improvement in quality, efficiency, and performance. NCAs equipped with our implicit decoder can generate full-HD outputs in real time while preserving their selforganizing, emergent properties. Moreover, because each MLP processes cell states independently, inference remains highly parallelizable and efficient. We demonstrate the applicability of our approach across multiple NCA variants and multiple tasks, including texture generation and morphogenesis (growing patterns from a seed), showing that with our proposed framework, NCAs seamlessly scale to high-resolution outputs with minimal computational overhead.


## Submission type: Late Breaking Abstracts

## Introduction

Complex self-organizing systems consist of numerous simple components that interact through simple ${ }^{1}$ local $^{2}$ rules,

[^3]producing coherent macro behavior without centralized control. In nature, such systems manifest across many scales: elementary particles bind into atoms and molecules, which then assemble into the diverse materials we observe; A single fertilized cell undergoes differentiation to form a fully developed organism during morphogenesis; and neurons coordinate and synchronize their activations to support coherent cognitive functions. In all cases, the global structure emerges not from top-down planning but from the collective effect of countless simple local interactions.

Early computational explorations of self-organization trace back to Alan Turing's reaction-diffusion model of morphogenesis (Turing, 1952) and John von Neumann's formulation of cellular automata (Neumann, 1966). Both lines of work revealed that simple hand-crafted local rules, applied through time, can yield intricate global patterns, and they opened a rigorous computational pathway for studying complex systems in biology, physics, and computer science. Neural Cellular Automata (NCAs) marked a turning point, addressing a key limitation of earlier hand-designed rules by integrating neural networks into self-organizing systems and enabling data-driven discovery of local rules (Mordvintsev et al., 2020). Once trained, an NCA can grow complex shapes or images from a single seed (Mordvintsev et al., 2020; Sudhakaran et al., 2021) or create rich patterns and textures (Niklasson et al., 2021; Pajouheshgar et al., 2023, 2024a,b). Because it learns an underlying self-organizing process rather than a direct mapping, the model inherits many distinctive features of natural self-organizing systems: robustness to perturbations and the ability to heal damaged regions (Mordvintsev et al., 2020), generalization to unseen environments such as different grid resolutions or mesh topologies (Pajouheshgar et al., 2024a), and emergent lifelike spontaneous motion (Xu et al., 2024), all while encoding the entire process with a lightweight, compute-efficient neural network.

In practice, current NCAs are trained on relatively lowresolution lattices containing roughly $10^{4} \sim 10^{5}$ cells, which translates to spatial resolutions of about $64 \times 64$ to $256 \times 256$ depending on the space dimensionality. This lim-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-079.jpg?height=383&width=839&top_left_y=269&top_left_x=155)
Figure 1: Sample Output of Our Hybrid Model. The NCA evolves on a coarse $128 \times 128$ lattice while our Local Pattern Producing Network (LPPN) renders an RGB image at $1024 \times 1024$ and, without retraining, at $8192 \times 8192$ in the magnified inset.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-079.jpg?height=511&width=844&top_left_y=904&top_left_x=150)
Figure 2: Summary of result. Our proposed method enables NCA to generate high-quality outputs with minimal extra cost. Our method is applicable to different NCA architectures and different training targets. Left: Growing 2D shapes and images from a single seed; Middle: Texture synthesis in 2D; Right: Texture synthesis on 3D Meshes. Demos are available at https:// cells2pixels.github.io.

itation arises from a combination of multiple practical and architectural constraints. First, training time and memory scale quadratically with cell grid size, quickly exhausting computational resources. Second, naive attempts to scale the grid quickly run into a communication bottleneck. Information propagates only one cell per update, so larger grids require many more steps for distant cells to communicate, which hinders model training and convergence. Third, although one could try to accelerate communication with larger convolution kernels or global operations, such fixes either run too slowly on edge devices or undermine the locality that gives NCAs their unique self-organizing properties. Consequently, it remains unclear whether NCAs can produce high-quality, high-resolution outputs while retaining their computational efficiency and characteristic emergent properties.

To overcome these limitations, and drawing on recent ad-
vances in implicit neural representations (INRs) (Stanley, 2007; Mildenhall et al., 2020; Müller et al., 2022), we pair NCA with a lightweight coordinate-based decoder, thereby decoupling NCA lattice size from output resolution. INRs were initially conceived as an approach to abstract away both time and locality in modeling embryonic pattern formation, generating an entire structure with a single neural network whose hidden layers correspond to successive developmental stages (Stanley, 2007). By combining NCAs and INRs, we adopt a hybrid approach: The NCA maintains an explicit, modest-sized grid of cells that evolve through local interactions, gradually producing a coarse feature map ${ }^{3}$. An Local Pattern Producing Network (LPPN), shared by all cells, then takes a locally averaged cell state together with continuous localized coordinates and produces the output color at any arbitrary position between the cells. From the biology perspective, the NCA provides the high-level blueprint that determines where structure should emerge, while the LPPN sidesteps the intricate biochemical refinement processes and instead renders those details in a single pass. This division of labor preserves the NCA's locality and emergent self-organizing properties, while enabling the model to scale seamlessly from coarse cell grids to finegrained, high-resolution outputs, as shown in Figure 1.

We train the NCA and LPPN jointly, end to end. The recurrent NCA, with only a few thousand trainable weights, still evolves on its coarse grid while the lightweight LPPN (adding roughly $20 \sim 30 \%$ more parameters) is invoked only to render high-resolution outputs for the loss evaluation. Because all recurrent updates happen at low resolution, training remains memory-efficient and fast, yet the added LPPN greatly increases the model's capacity. The LPPN is a tiny Multi-Layered Perceptron (MLP) applied independently to every cell. Thus, inference remains fully parallelizable on modern GPUs and fast even on edge devices. We apply our method to the two most common NCA tasks: growing shapes or images from a seed (Mordvintsev et al., 2020; Sudhakaran et al., 2021) and texture synthesis (Niklasson et al., 2021; Pajouheshgar et al., 2023, 2024a,b; Wang et al., 2025). To supervise these tasks at high resolution, we propose tailored loss functions that add only minimal memory or computation overhead. We demonstrate that our hybrid framework adapts effortlessly to NCAs operating on diverse spatial domains, from 2D grids (Mordvintsev et al., 2020; Niklasson et al., 2021; Pajouheshgar et al., 2023, 2024b) to 3D voxel grids (Larsson et al., 2025; Sudhakaran et al., 2021; Wang et al., 2025) and 3D meshes (Pajouheshgar et al., 2024a). Across all these settings, our model retains the characteristic self-organizing properties of NCAs, along with their efficiency and interactive controllability. Representative examples of the results are shown in Figure 2.

[^4]
## References

Larsson, M., Yamaguchi, H., Pajouheshgar, E., Shen, I.-C., Tojo, K., Chang, C.-M., Hansson, L., Broman, O., Ijiri, T., Shamir, A., Jakob, W., and Igarashi, T. (2025). The mokume dataset and inverse modeling of solid wood textures. ACM Transactions on Graphics, 44(4):18 pages.

Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. (2020). Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, pages 405-421.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing neural cellular automata. Distill. https://distill.pub/2020/growing-ca.

Müller, T., Evans, A., Schied, C., and Keller, A. (2022). Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1-15.

Neumann, J. v. (1966). Theory of self-reproducing automata. Mathematics of Computation, 21:745.

Niklasson, E., Mordvintsev, A., Randazzo, E., and Levin, M. (2021). Self-organising textures. Distill, 6(2):e00027-003.

Pajouheshgar, E., Xu, Y., Mordvintsev, A., Niklasson, E., Zhang, T., and Süsstrunk, S. (2024a). Mesh neural cellular automata. ACM Trans. Graph.

Pajouheshgar, E., Xu, Y., and Süsstrunk, S. (2024b). Noisenca: Noisy seed improves spatio-temporal continuity of neural cellular automata. volume ALIFE 2024: Proceedings of the 2024 Artificial Life Conference of Artificial Life Conference Proceedings, page 57.

Pajouheshgar, E., Xu, Y., Zhang, T., and Süsstrunk, S. (2023). Dynca: Real-time dynamic texture synthesis using neural cellular automata. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2074220751.

Stanley, K. O. (2007). Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines, 8(2):131-162.

Sudhakaran, S., Grbic, D., Li, S., Katona, A., Najarro, E., Glanois, C., and Risi, S. (2021). Growing 3d artefacts and functional machines with neural cellular automata. In 2021 Conference on Artificial Life.

Turing, A. (1952). The chemical basis of morphogenesis. Philosophical Transactions of the Royal Society of London Series B, 237(641):37-72.

Wang, D., Pajouheshgar, E., Xu, Y., Zhang, T., and Süsstrunk, S. (2025). Volumetric temporal texture synthesis for smoke stylization using neural cellular automata. arXiv preprint arXiv:2502.09631.

Xu, Y., Pajouheshgar, E., and Süsstrunk, S. (2024). Emergent dynamics in neural cellular automata. volume ALIFE 2024: Proceedings of the 2024 Artificial Life Conference of Artificial Life Conference Proceedings, page 96.

# Petri Dish NCA 

Ivy Zhang ${ }^{1}$, Sebastian Risi ${ }^{1}$, and Luke Darlow ${ }^{1}$<br>${ }^{1}$ Sakana AI, Japan<br>luke @ sakana.ai


#### Abstract

Artificial life simulations in search of open-endedness are often based on artificial evolution. While such systems can generate complexity, gradient-based learning remains underexplored and may offer a complementary path to openendedness. We introduce Petri Dish Neural Cellular Automata (PD-NCA): a differentiable multi-agent substrate consisting of a competitive population of neural cellular automata (NCA), trained continuously as an artificial life simulation. Differentiable PD-NCA enable end-to-end learning in a competitive, multi-agent system, which we hypothesize can induce open-ended complexification. Importantly, and unlike typical NCA experiments, the models in our experiments are continuously learning through gradient descent. Exploratory experiments demonstrate that PD-NCA show signs of emergent behavioral complexity and cooperation. More broadly, our work introduces a new substrate where learning could potentially combine with evolution to form open-ended systems.


## Submission type: Late Breaking Abstract

## Introduction

The field of artificial life (ALife) has seen much focus on evolutionary algorithms for optimization. Evolution holds significant potential for open-endedness [7], as it is not limited by the requirement for the underlying models, substrate, or techniques to be differentiable. That being said, recent
work in differentiable systems have shown promise for ALife simulation $[5,8,6,1,4]$.

Neural Cellular Automata (NCA) [5] are convolutional neural networks (CNNs) that learn local update rules, using backpropagation, to yield growth-like behavior in a 2D substrate. NCA demonstrated that backpropagation could learn complex local update rules, but were limited in two key ways: (1) they were single-agent-based, and (2) their loss functions were not designed for open-ended evolution.

We introduce Petri Dish NCA (PD-NCA), a framework that extends NCA to multi-agent scenarios where individuals compete for growth within a shared differentiable substrate. Unlike typical NCA experiments with fixed objectives, PD-NCA have no train-test split: models are continuously optimized throughout the simulation, making gradient-based learning part of the dynamics itself. This approach may prove more scalable than pure evolution, as backpropagation can efficiently optimize millions of parameters and benefits from neural scaling laws [2] that predict improved performance with increased model size.

In this paper we explore whether this combination of competition and continuous learning produces emergent ALife phenomena. We present our methods and initial evidence of spontaneous cooperation arising between competing agents. Our implementation is available here to enable further exploration by the ALife research community.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-082.jpg?height=440&width=1726&top_left_y=1966&top_left_x=171)
Figure 1: Signs of emergent complexity in PD-NCA: (a) structured pairing of two NCA showing signs of symbiosis, (b) stable continuous competition of several NCA, and (c) wave-like spiral patterns when using three NCA. These frames are of the simulation are not necessarily evenly spaced, but the arrow of time is from left to right. We share multiple videos here.

## Methods

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-083.jpg?height=483&width=700&top_left_y=351&top_left_x=221)
Figure 2: Competition mechanism between 2 NCA: (1) Cosine similarity between attack and defense are summed to form strength; (2) Strengths are passed through softmax to create relative update weight and new aliveness distribution.

Our simulation environment is a 2D grid where multiple NCA coexist and compete. A cell in the grid is defined by a state vector containing C channels, partitioned into 'attack', 'defense', and hidden channels. Fig. 2 shows how the attack and defense vectors interact between any two NCA. At each timestep, the simulation proceeds through three distinct phases: processing, competition, and state update.
Processing Each NCA is a CNN that 'views' a small region when proposing updates for a cell. These updates are masked by an aliveness value, allowing only updates to alive cells and their neighbors. Each NCA's aliveness is stored as a channel in the grid (hidden from the NCA). We also apply a static environment vector to provide a constant background update proposal, essentially bootstrapping growth and allowing for 'environmental' changes in future work.
Competition Proposed updates are resolved through a strength-based weighting system. At every cell, each NCA's strength is the sum of the cosine similarities between its attack channels and opposing defense channels. For example, consider a cell where $\mathrm{NCA}_{A}$ and $\mathrm{NCA}_{B}$ both propose updates. $\mathrm{NCA}_{A}$ 's strength would be $\left\langle\mathrm{att}_{A}, \operatorname{def}_{B}\right\rangle+\left\langle\mathrm{att}_{A}, \operatorname{def}_{\text {env }}\right\rangle$, while NCA ${ }_{B}$ 's strength would be $\left\langle\operatorname{att}_{B}, \operatorname{def}_{A}\right\rangle+\left\langle\operatorname{att}_{B}, \operatorname{def}_{\text {env }}\right\rangle$. Strengths are normalized via softmax to determine each NCA's relative contribution to the final update.
State update The final delta 'update' to the grid is the sum of the weighted proposed updates from each NCA and the background environment vector. The aliveness channels per cell are set to the relative strengths for each NCA. Fig. 3 shows the dominant aliveness at various snapshots throughout simulation. Any NCA with aliveness below a threshold has their aliveness redistributed among the remaining NCA.

NCA optimize for growth by maximizing their total aliveness across the grid. The result is that NCA grow outwards
while also having to compete for space. We apply the logarithm function to an NCA's summed aliveness to stablize training. The result is that each NCA tries to minimize $L_{i}=-\log \left(\Sigma_{x, y} A_{i}(x, y)\right) ; A_{i}(x, y)$ is the $\mathrm{NCA}_{i}$ 's aliveness at position ( $x, y$ ).

## Preliminary Results

We trained PD-NCAs using CNNs (up to 3 layers, 128 channels, $\approx 500 K$ parameters), and up to 15 NCA on $256 \times 256$ grids. Fig. 1 shows selected frames from some simulations. Fig. 1(a) shows a simulation where structure emerges between groups of NCA (e.g., cyan-purple and blue-orange), giving credence to the potential of PD-NCAs as an ALife simulation. Fig. 1(b) and (c) both show patterns akin to chemical wave propagation and oscillatory dynamics. As further evidence, Fig. 3 shows the rise and fall of two distinct groups of NCAs.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-083.jpg?height=213&width=823&top_left_y=1069&top_left_x=1071)
Figure 3: Territory size dynamics showing oscillations and cooperation between NCA pairs.

The complex dynamics of PD-NCA simulations are best understood through video demonstrations where structures emerge and disappear as the simulation unfolds. These dynamics arise because the underlying NCA are trained continuously throughout the simulation, learning to adapt insitu as their environment changes. This adaptive behavior grew richer on larger grids, suggesting that scaling beyond $256 \times 256$ grids could yield even more complex dynamics.

## Extensions

We plan to integrate evolution to create a hybrid system where learning and evolution operate simultaneously. For example, when NCA split spatially, the fragments could become independent lineages associated with their own optimizers. This would effectively lift the constraint on the number of alive NCA, thus enabling both gradient-based adaptation and evolutionary dynamics to work in tandem. Such a hybrid approach is designed to leverage the complementary strengths of gradient descent's efficiency when training CNNs and evolution's capacity for open-ended exploration.

Beyond individual growth objectives, we plan to explore system-wide optimization goals (e.g. compressibility) that leverage PD-NCA's fully differentiable architecture. Tools like ASAL [3] could automatically discover novel substrate parameterizations that produce even richer dynamics.

## References

Etienne Guichard, Felix Reimers, Mia Kvalsund, Mikkel Lepperød, and Stefano Nichele. Arc-nca: Towards developmental solutions to the abstraction and reasoning corpus. arXiv preprint arXiv:2505.08778, 2025.

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling Laws for Neural Language Models, January 2020. URL http://arxiv.org/ abs/2001.08361. arXiv:2001.08361 [cs].

Akarsh Kumar, Chris Lu, Louis Kirsch, Yujin Tang, Kenneth O. Stanley, Phillip Isola, and David Ha. Automating the Search for Artificial Life with Foundation Models, May 2025. URL http://arxiv.org/abs/2412.17799. arXiv:2412.17799 [cs].

Pietro Miotti, Eyvind Niklasson, Ettore Randazzo, and Alexander Mordvintsev. Differentiable Logic Cellular Automata: From Game of Life to Pattern Generation, June 2025. URL http : //arxiv.org/abs/2506.04912. arXiv:2506.04912 [cs].

Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, and Michael Levin. Growing neural cellular automata. Distill, 2020. doi: 10.23915/distill.00023.

Ettore Randazzo and Alexander Mordvintsev. Simulating an artificial biome of plants with biomaker ca. In Artificial Life Conference Proceedings 36, volume 2024, page 120. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA, 2024.

Kenneth O Stanley, Joel Lehman, and Lisa Soros. Open-endedness: The last grand challenge you've never heard of. While openendedness could be a force for discovering intelligence, it could also be a component of AI itself, 2017.

Shyam Sudhakaran, Djordje Grbic, Siyan Li, Adam Katona, Elias Najarro, Claire Glanois, and Sebastian Risi. Growing 3d artefacts and functional machines with neural cellular automata. In Artificial Life Conference Proceedings 33, volume 2021, page 108. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA, 2021.

# Embodied Classification with Neural Cellular Automata 

Iliya Zhechev<br>Sofia University, Bulgaria, izhechev @uni-sofia.bg


#### Abstract

We present an approach to supervised classification using Neural Cellular Automata (NCA), where classification emerges through localized pattern formation. Our model learns to classify inputs by transforming the initial grid state into a stable attractor in a designated region. The system reliably converges to distinct class-specific attractors through purely local interactions, without centralized control. We evaluate our model on the MNIST classification benchmark and show that it achieves competitive performance. Our results highlight the potential of NCAs as a substrate for decentralized and embodied computation in artificial life systems.


Submission type: Late Breaking Abstracts
Code available at: https://anonymous.4open. science/r/nca-classification-8A19

## Introduction

Cellular Automata (CA) are a classic model of decentralized computation, where complex dynamics emerge from simple, local interactions (von Neumann, 1966). NCAs, a differentiable variant trained using gradient descent, have demonstrated remarkable capabilities in tasks such as morphogenesis, self-repair (Mordvintsev et al., 2020; Niklasson et al., 2021), texture synthesis and classification (Randazzo et al., 2020).

In this work, we explore an approach to image classification where the model learns to activate a specific circular region on the grid corresponding to the input class. Each class is mapped to a distinct, spatially localized activation pattern, allowing classification to be performed by sampling the activation signal at a fixed location at the end of the simulation.

Rather than encoding class identity as a scalar value in a hidden channel-as in Randazzo et al. (2020)-the outcome of the classification emerges as a localized pattern on the grid, echoing the way biological systems use spatial cues for signaling.

We evaluate the models on the MNIST dataset (Deng, 2012), demonstrating that it achieves competitive accuracy while preserving hallmark features of CA-based systems:

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-086.jpg?height=835&width=828&top_left_y=842&top_left_x=1078)
Figure 1: The diagram provides a visual overview of how the region-activation model performs classification by selectively activating class-specific areas of the grid.

decentralization and self-organization. Our results suggest that NCAs can serve as viable substrates for parallel classification tasks, offering a step toward more bio-inspired machine learning architectures.

In this report, we use the term embodied computation to describe a system in which computation emerges through local interactions that unfold in space and time, producing visible patterns that can be directly observed and interpreted.

## Methodology

Our model follows the architecture of Mordvintsev et al. (2020). Each cell in the $80 \times 80$ NCA grid maintains a 16channel state. For each channel, we compute spatial features using two Sobel filters (horizontal and vertical) and an iden-
tity kernel. The resulting values are then processed by a twolayer fully connected network, which defines the cell update rule and contains the parameters optimized during training.

The readout patter is placed equidistantly around the central digit to ensure uniform information transfer and avoid overlap with the input (Figure 2). Since the NCA is initialized as an identity function, alive-masking suppresses all cells in the output regions, blocking gradient flow and preventing learning. We therefore disable alive-masking in our implementation.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-087.jpg?height=260&width=259&top_left_y=730&top_left_x=443)
Figure 2: Output readout classification pattern. Each of the circles coresponds to the output activation location of each of the 10 classification labels.

At each training step, we compute the mean-squared error (MSE) loss between the target circular activation pattern and the final state of the NCA rollout, which consists of a randomly sampled number of steps between 30 and 45 . Varying the rollout length during training encourages the system to converge toward a stable attractor state by the end of the simulation.

After training for 100000 steps using the Adam optimzer (Kingma and Ba, 2017), we obtain a model capable of classification by reading out the activation values from all predefined output regions and selecting the class corresponding to the region with the highest average activation, as illustrated in Figure 1.

## Results

Figure 3 present a batch of NCA rollouts. Each example illustrates how the NCA transforms an input digit into a spatial activation pattern, where one of the predefined circular regions is selectively activated based on the class.

Our approach achieves relatively high train and test accuracy, 0.9247 and 0.9136 respectively. With the two values remaining close-indicating strong generalization.

## Consclusion

In this work we presented a dynamic classification approaches using NCAs leveraging spatially grounded mechanisms for decision-making. This work reinforces the potential of NCAs as a platform for bio-inspired computation and opens new avenues for combining morphogenetic dynamics with structured, task-driven behavior.

While our model demonstrate that NCAs can effectively perform classification through spatial patterning, the ap-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-087.jpg?height=965&width=794&top_left_y=274&top_left_x=1096)
Figure 3: A representative sample of successful rollouts for each classification class. Each row depicts the time progression horizontally.

proach rely on carefully designed readout structures and training objectives to encourage information spread. In future work, we aim to explore more principled and less contrived mechanisms for encouraging efficient information propagation.

## References

Deng, L. (2012). The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-142.

Kingma, D. P. and Ba, J. (2017). Adam: A method for stochastic optimization.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing neural cellular automata. Distill. https://distill.pub/2020/growing-ca.

Niklasson, E., Mordvintsev, A., Randazzo, E., and Levin, M. (2021). Self-organising textures. Distill. https://distill.pub/selforg/2021/textures.

Randazzo, E., Mordvintsev, A., Niklasson, E., Levin, M., and Greydanus, S. (2020). Self-classifying mnist digits. Distill. https://distill.pub/2020/selforg/mnist.
von Neumann, J. (1966). Theory of Self-Reproducing Automata. University of Illinois Press, Urbana and London.

# Rosetta: Real-Time Bio-Hybrid Interface for Protist Collectives 

Charles Fosseprez ${ }^{1}$,<br>${ }^{1}$ Mathematical and Physical Ethology laboratory, Hokkaido university, Sapporo, Hokkaido, Japan charles.fosseprez.pro@gmail.com


#### Abstract

This paper presents the technical validation of Rosetta, an open-source platform for real-time, closed-loop experiments with protist populations, with biological experiments currently underway. Rosetta integrates high-resolution tracking of hundreds of Paramecium with individual precision lightbased feedback at 15 FPS, enabling population behaviors to directly influence environmental feedback. This provides a reproducible platform for future bio-hybrid experiments at both single-cell and collective levels.


## Submission type: Late Breaking Abstracts

## Introduction

Protists-single-celled eukaryotic organisms-offer unique advantages for studying bio-hybrid control systems. Despite lacking nervous systems, these organisms demonstrate remarkable computational abilities: Paramecium exhibits nonneural learning (Alipour et al., 2018; Brette, 2021), while amoebae solve spatial problems (Nakagaki et al., 2000) and display associative memory via metabolic network dynamics analogous to Hopfield networks (De la Fuente et al., 2013, 2019). At population scales, hundreds of these "swimming neurons" interact physically and chemically to produce emergent collective behaviors without centralized coordination.

Particularly relevant for experimental control are the natural phototactic behaviors exhibited by many protist species. These organisms display diverse responses to light: For example, some Paramecium species accumulate near light sources (Iwatsuki and Naitoh, 1988), while others strongly react to illumination (Iwatsuki and Naitoh, 1983). These varied responses to light stimuli provide a natural interface for investigating how sensory information percolates through populations and shapes collective dynamics.

However, tools for systematic experimentation remain limited; microscopy allows observation but not intervention, while commercial photo-stimulation systems are costly and lack integrated feedback. Recent open-source work Rubio Denniss et al. (2022) demonstrated proof-of-concept
closed-loop control, but lack the robustness and scalability for reproducible population-scale research. Rosetta addresses this gap by enabling production-ready real-time bidirectional coupling between protist populations and their environment. The system delivers precisely controlled spatiotemporal light stimuli while quantifying collective responses through behavioral and topological metrics, enabling future investigations of emergent bio-hybrid dynamics in protist swarms as described in Figure 1.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-088.jpg?height=572&width=764&top_left_y=1247&top_left_x=1114)
Figure 1: Three-layer closed-loop architecture (physical, abstraction, artificial), enabling real-time feedback based on population dynamics. The loop operates in real time at up to $15 \sim$ FPS with $10 \sim \mu \mathrm{~m}$ projection resolution on $13 \sim \mathrm{MP}$ images across a $3 \sim \mathrm{~cm}$ field of view, with integrated deep learning-based detection and tracking, a combination of capabilities not available in commercial systems.

## System Design and Implementation

The hardware, costing approximately $\$ 3,000$, relies on 3D printed parts and off-the-shelf components. It integrates a large field-of-view microscope (up to 3 cm , depending on magnification) using a high-resolution, high-FPS industrial camera, together with a DMD projection system delivering high-resolution light ( $10 \mu \mathrm{~m}$ resolution) across 3 color chan-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-089.jpg?height=337&width=849&top_left_y=269&top_left_x=150)
Figure 2: Detection speed and accuracy benchmark. Left: median processing speed ( ± std) over 1,000 frames with 100 detections per frame for three image sizes on a consumergrade NVIDIA GPU. Blue: PyTorch YOLOv8-Nano; green: TensorRT engine; orange: optimized SwarmTracker. Right: accuracy metrics show optimization maintains detection quality. With a 50 ms latency on high resolution images (essential for detecting hundreds of individuals simultaneously), the system allows real-time detection on an arena with hundreds of protists.

nels. The optical path was specifically designed for this system to achieve these performance levels at a reasonable cost, Software-based calibration and distortion mapping further enhance projection precision. It enables precise feedback targeting of individual protists or specific regions within a population. The system uses one color (usually red, $\lambda=650 \mathrm{nm})$ for low-light imaging and two colors for stimulation. A consumer-grade CPU and CUDA-compatible GPU is required, either through a computer or, for lower-performance requirements, a Jetson Orin (included in the total price).

The platform's software integrates into the SwarmMaster pipeline, which handles detection, tracking, abstraction, agent control, and feedback through a plugin architecture that supports flexible integration of analysis pipelines and control strategies, enabling rapid experimentation with a range of computational overlays. Tracking is performed in a highly optimized tracking-by-detection pipeline called SwarmTracker that uses a YOLOv8 model optimized for high-resolution images (13MP), the performances results are presented in Figure 2.

The tracker (SwarmSORT) was specifically developed to operate at the required frequency, as available solutions did not meet performance nor precision requirements (tracker code and demo: https://github.com/ cfosseprez/swarmsort), the performances results are presented in Figure 3. The population analysis/embedding component integrates behavioral and topological metrics (online topology analysis code and demo: https:// github.com/cfosseprez/graphizy). Currently, published modular software elements are available, while the rest of the SwarmTracker pipeline is awaiting review. Full 3D models and machine components will be published at the same time as the integration of the feedback into the

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-089.jpg?height=330&width=844&top_left_y=269&top_left_x=1073)
Figure 3: Tracking scalability. Tracking speed remains real-time up to 500 objects ( $\sim 20$ FPS), even when using appearance embeddings (Magenta line) for improved performances. The tracking algorithms sustain high accuracy across all densities, achieving MOTA scores above 0.9 even in crowded scenarios with live paramecium, demonstrating the robustness and efficiency of the system.

SwarMaster pipeline with experimental data in a second issue when the experimental cycle will begin.

An early-stage demonstration video (epilepsy warning contains flashing lights) shows feedback control with a small Paramecium population in an 8 mm arena using a previous version of the system. In this case a simple speed threshold triggers the stimulation of a precise individual. Preliminary tests confirm phototactic responses in most of the individuals. Paramecium's phototactic responses are welldocumented in literature. The projected feedback can be adapted in size and intensity. In standard operation optical filters prevent stimulation light from reaching the sensor. The projection operates within milliseconds, ensuring accurate individual targeting. The system delivers projections with $10 \mu \mathrm{~m}$ spatial resolution and targeting accuracy within $5 \mu \mathrm{~m}$. Comprehensive feedback response data will accompany future publications with the complete integrated system.

## Future Directions: Encoding Information in Light Landscapes

The results presented in this work establish the basic feasibility of light-mediated cybernetic coupling with protist collectives. However, our current demonstration uses only simple feedback. These proof-of-concept results reveal promising avenues for developing more sophisticated artificial control layers. Rosetta can be used simply to assess learning abilities in Paramecium. Future work could explore how different computational approaches might generate more complex light landscapes. For instance, cellular automata could create spatially propagating patterns, while reactiondiffusion systems might generate continuously evolving gradients. More advanced approaches like neural networks could potentially learn to generate optimal stimulus patterns based on protist behavioral feedback.

## Acknowledgements

This work was supported by the Sasakawa Science Research grant No. 2023-5025.

## References

Alipour, A., Dorvash, M., Yeganeh, Y., and Hatam, G. (2018). Paramecium learning: New insights. The Journal of protozoology research, 28(1-2):22-32.

Brette, R. (2021). Integrative neuroscience of paramecium, a "swimming neuron". Eneuro, 8(3).

De la Fuente, I. M., Bringas, C., Malaina, I., Fedetz, M., CarrascoPujante, J., Morales, M., Knafo, S., Martínez, L., PérezSamartín, A., López, J. I., et al. (2019). Evidence of conditioned behavior in amoebae. Nature communications, 10(1):3690.

De la Fuente, I. M., Cortes, J. M., Pelta, D. A., and Veguillas, J. (2013). Attractor metabolic networks. PLoS One, 8(3):e58284.

Iwatsuki, K. and Naitoh, Y. (1983). Behavioral responses in paramecium multimicronucleatum to visible light. Photochemistry and Photobiology, 37(4):415-419.

Iwatsuki, K. and Naitoh, Y. (1988). Behavioural responses to light in paramecium bursaria in relation to its symbiotic green alga chlorella. Journal of experimental biology, 134(1):43-60.

Nakagaki, T., Yamada, H., and Tóth, Á. (2000). Maze-solving by an amoeboid organism. Nature, 407(6803):470-470.

Rubio Denniss, A., Gorochowski, T. E., and Hauert, S. (2022). An open platform for high-resolution light-based control of microscopic collectives. Advanced Intelligent Systems, 4(7):2200009.

# Action and Perception in Collective Motion: The Role of Active Sensing and Minimization of Prediction Error 

Nicolas Bessone ${ }^{1}$, Heiko Hamann ${ }^{2}$, and Andreagiovanni Reina ${ }^{2,3}$<br>${ }^{1}$ IT University of Copenhagen, Denmark<br>${ }^{2}$ Centre for the Advanced Study of Collective Behaviour, Universität Konstanz, Germany<br>${ }^{3}$ Department of Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany<br>${ }^{1}$ nbes@itu.dk


#### Abstract

We investigate how internal sensor modulation-the adaptive control of sensor usage probabilities-shapes collective behavior in multi-agent systems. In our framework, agents adjust both their actions and sensing strategies to minimize prediction error, following principles from predictive coding, active inference and active sensing. Through multi-agent simulations, we show that active sensor modulation improves coordination, reduces prediction error, and enables more efficient sensor usage compared to passive baselines. These findings highlight active sensing as a scalable mechanism for organizing collective behavior under uncertainty.


## Submission type: Late Breaking Abstracts

Code available at: https://anonymous.4open. science/r/AIBoids-FE31/

## Introduction, Background and Related Works

How distributed systems self-organize under uncertainty is a central question across biology, robotics, and artificial life. The Free Energy Principle (FEP) posits that agents act to minimize variational free energy-a bound on surprise about their sensory inputs (Friston, 2008; Friston et al., 2010; Bogacz, 2017). This involves optimizing internal models and sampling the environment to reduce prediction error. Predictive Coding ( $\boldsymbol{P C}$ ) realizes this via hierarchical inference (Rao and Ballard, 1999; Denham and Winkler, 2020), while Active Inference (AI) extends it to action selection (Friston et al., 2010). Active Sensing (AS) reduces uncertainty by controlling sensory acquisition, either through overt physical movement to gather information (AS1), or through internal modulation of sensor usage (AS2). Prior research has largely focused on AS1, where agents move to expose sensors to new information, but the role of AS2-adaptive modulation of sensors usage-remains underexplored. In this work, we address this gap by focusing on AS2, where agents dynamically modulate their own sensors. For clarity, we treat fixed sensing strategies as passive sensing (PS) baselines.

Empirical and simulated studies show these mechanisms drive rich behaviors. Minimal $A I$ agents show photo-
taxis via prediction error minimization without task-specific goals (Baltieri and Buckley, 2017); evolved agents trained on prediction-based objectives display flocking, dispersion, and robust collective patterns (Borkowski and Hamann, 2017; Hamann, 2014). Surprise minimization also guides self-assembly and adaptation in hardware and simulation (Kaiser and Hamann, 2022), and explains animal-like group dynamics without explicit coordination (Heins et al., 2024). These findings support prediction error minimization as a general principle for emergent behavior in both natural and artificial collectives.

## Method

We adopt Reynold's Boids framework (Reynolds (1987)) for agent kinematics and local perception, with agents implementing $P C, A I$, and $A S$. Each agent (see details in Fig. 1) maintains a state $S=[O, V, P]$ composed of external observations, velocity, and per-sensor sampling probabilities. A shared neural network $f_{\theta}$ predicts the next sensory state and issues control updates:

$$
\begin{gathered}
\hat{S}_{t+1}^{i}, \Delta V_{t}^{i}, \Delta P_{t}^{i}=f_{\theta}\left(S_{t}^{i}\right) \\
V_{t+1}^{i}=V_{t}+\Delta V_{t}^{i} \quad ; \quad P_{t+1}^{i}=P_{t}^{i}+\Delta P_{t}^{i}
\end{gathered}
$$

Perception is stochastic: each sensor fires if its probability $P>R \sim \mathcal{U}(0,1)$. Inactive sensors retain their last value, providing short-term memory. After acting and sensing, the agent realizes $S_{t+1}$. Training is centralized, with shared parameters $\theta$ and per-agent $P$, minimizing prediction error $\mathcal{L}_{t}$ across batch $B_{t}$ with learning rate $\eta$ :

$$
\mathcal{L}_{t}=\frac{1}{\left|B_{t}\right|} \sum_{i \in B_{t}} \mathcal{L}_{t}^{i} \quad ; \quad \theta_{t+1} \leftarrow \theta_{t}-\eta \nabla \mathcal{L}_{t}
$$

## Experiments

We use multi-agent simulations on a toroidal map to study the impact of $A S$ on collective behavior. Agents interact via elastic collisions conserving energy and momentum. We vary key parameters: hidden neural network nodes $N_{h} \in\{2, \ldots, 128\}$, number of external sensors $N_{e} \in\{2, \ldots, 128\}$ among others. Each configuration is repeated 10 times with randomized initial conditions.

Baseline comparisons are made with Passive Sensing (PS), using fixed sensor probabilities: PS: 1 (all set to 1 ) and

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-093.jpg?height=451&width=837&top_left_y=274&top_left_x=155)
Figure 1: Schematic of an agent's internal control loop. At each step, the agent processes its state $S_{t}=\left[O_{t}, V_{t}, P_{t}\right]$, where $O \in[0,1]^{N_{e}}$ are exteroceptive sector readings, $V=\left[v_{x}, v_{y}\right]$ the velocity, and $P \in[0,1]^{N_{e}+2}$ the sensors sampling probabilities. The network $\mathrm{NN}_{\theta}$ predicts the next state $\hat{S}_{t+1}$ and outputs action ( $\Delta V$ ) and sensing control ( $\Delta P$ ). Sensors fire stochastically when $P>R$ with $R \sim \mathcal{U}(0,1)$. Learning is driven by the predictive loss $\mathcal{L}=\left|S_{t+1}-\hat{S}_{t+1}\right|^{2}$.

PS: $R$ (random in $\mathcal{U}(0,1)$ ). We evaluate collective behavior using four metrics: Polarization, which measures alignment in agents' movement directions; Bernoulli Entropy, which captures uncertainty in sensor activation; Sensor Usage, quantifying the average proportion of active sensors per agent; and Prediction Error, assessing the gap between expected and actual observations under the predictive coding framework. Together, these metrics characterize coordination, sensing dynamics, and internal model accuracy.

## Results and Conclusions

Figure 2 compares behavioral outcomes across sensing strategies. AS achieves the strongest alignment and lowest prediction error while using fewer sensors. Polarization emerges not as an explicit objective, but as a byproduct of agents stabilizing their sensory inputs through coordinated motion.

Figure 3 shows how these effects vary with predictive capacity. Under limited resources, agents rely on alignment to manage uncertainty; with greater capacity, they reduce error individually and coordination weakens. Crucially, full sensing (PS:1) does not outperform selective sensing, showing that structuring perception-rather than maximizing in-put-supports more accurate predictions.

Together, these results demonstrate that sensor modulation (AS2) improves coordination, predictive accuracy, and sensing efficiency compared to passive strategies.

Future work could investigate decentralized learning, where each agent maintains its own parameters rather than sharing a global model, to explore how individual adaptation shapes collective dynamics.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-093.jpg?height=766&width=678&top_left_y=315&top_left_x=1155)
Figure 2: Distributions of behavioral metrics across all runs, comparing AS with PS: 1 and PS:R baselines. AS agents achieve highest polarization ( $\mu=0.91$ ), lowest prediction error ( $\mu=0.09$ ), and low entropy ( $\mu=0.03$ ), reflecting aligned motion, accurate prediction, and selective sensing. PS: 1 agents show zero entropy and full usage ( $\mu=1.0$ ), while $P S: R$ agents exhibit mean sensor usage $\mu=0.5$ and expected entropy $\mu \approx 0.72$, matching the theoretical values.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-093.jpg?height=520&width=830&top_left_y=1555&top_left_x=1078)
Figure 3: Polarization (top) and prediction error (bottom) over the last 10 simulation timesteps across sensing strategies and ( $N_{h}, N_{e}$ ) combinations. AS agents consistently combine higher polarization with lower error than passive baselines. Across all strategies, polarization is strongest under limited predictive capacity, suggesting alignment as a collective strategy for uncertainty. As predictive capacity increases, error reduction shifts from collective alignment to individual prediction, leading to weaker polarization.

## References

Baltieri, M. and Buckley, C. L. (2017). An active inference implementation of phototaxis. In Proceedings of the ECAL 2017, pages 36-43. MIT Press.

Bogacz, R. (2017). A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathematical Psychology, 76:198-211.

Borkowski, R. and Hamann, H. (2017). Evolving robot swarm behaviors by minimizing surprise: results of simulations in 2-d on a Torus. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO '17, pages 1679-1680, New York, NY, USA. Association for Computing Machinery.

Denham, S. L. and Winkler, I. (2020). Predictive coding in auditory perception: challenges and unresolved questions. European Journal of Neuroscience, 51(5):1151-1160. _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/ejn.13802.

Friston, K. (2008). Hierarchical Models in the Brain. PLOS Computational Biology, 4(11):e1000211. Publisher: Public Library of Science.

Friston, K. J., Daunizeau, J., Kilner, J., and Kiebel, S. J. (2010). Action and behavior: a free-energy formulation. Biological Cybernetics, 102(3):227-260.

Hamann, H. (2014). Evolution of collective behaviors by minimizing surprise, volume ALIFE 14: The Fourteenth International Conference on the Synthesis and Simulation of Living Systems of Artificial life conference proceedings. Pages: 344-351 tex.eprint: https://direct.mit.edu/isal/proceedings-pdf/alife2014/26/344/1901265/978-0-262-32621-6ch055.pdf.

Heins, C., Millidge, B., Da Costa, L., Mann, R. P., Friston, K. J., and Couzin, I. D. (2024). Collective behavior from surprise minimization. Proceedings of the National Academy of Sciences, 121(17):e2320239121. Publisher: Proceedings of the National Academy of Sciences.

Kaiser, T. K. and Hamann, H. (2022). Innate Motivation for Robot Swarms by Minimizing Surprise: From Simple Simulations to Real-World Experiments. IEEE Transactions on Robotics, 38(6):3582-3601. Conference Name: IEEE Transactions on Robotics.

Rao, R. and Ballard, D. (1999). Predictive Coding in the Visual Cortex: a Functional Interpretation of Some Extra-classical Receptive-field Effects. Nature neuroscience, 2:79-87.

Reynolds, C. W. (1987). Flocks, herds and schools: A distributed behavioral model. In Proceedings of the 14th annual conference on Computer graphics and interactive techniques, pages 25-34. ACM.

# Molecular Assembly Indices to study Natural Metabolic Evolution 

Jeppe Van den Langenbergh ${ }^{1}$, Carlos Vigil-Vásquez ${ }^{1,2}$, Bram Spanoghe ${ }^{1}$, Bernard De Baets ${ }^{1}$ and Michiel Stock ${ }^{1 \star}$<br>${ }^{1}$ KERMIT Research Unit, Department of Data Analysis and Mathematical Modelling, Ghent University, Ghent, Belgium<br>${ }^{2}$ SCNRS, Inserm, CHU Lille, Institut Pasteur de Lille, U1019 - UMR 9017 - CIIL - Center for Infection and Immunity of Lille, F-59000 Lille, France<br>★ michiel.stock@ugent.be


#### Abstract

Assembly Theory's Molecular Assembly (MA) index quantifies complexity based on an object's constructive history. To validate its biological relevance, we conducted a multi-scale analysis of over 7,500 metabolites from the KEGG database. Our results demonstrate that MA captures significant patterns at all organizational levels. At lower scales, MA reflects molecular features and tracks changes in complexity driven by different enzyme classes. At higher scales, pathway-level MA profiles distinguish core from specialized metabolism, aligning with evolutionary models. The MA distributions of entire metabolic repertoires contain strong taxonomic signals, successfully clustering major domains of life, such as animals and plants. These findings establish MA as a powerful, scalable metric for investigating the structure and evolution of biological complexity.


## Submission type: Late Breaking Abstract

## Background

Understanding the emergence of biological complexity, from molecular architecture to entire metabolic networks, remains a fundamental challenge in (artificial) biology. While evolutionary theory explains the persistence of successful traits, it offers limited predictive power regarding the generation of novelty. Similarly, the laws of physics alone cannot distinguish between functionally significant structures and random fluctuations, as they lack a functional view of the universe. To bridge this explanatory gap, Assembly Theory (AT) has been proposed as a framework that quantifies complexity based on the constructive history of an object, rather than just its static properties (Sharma et al., 2023; Jirasek et al., 2024). At its core, AT states that the complexity of an object can be measured by the length of its shortest construction pathway from a set of basic building blocks, critically allowing for the reuse of intermediate structures in a recursive process. The application of this concept to molecules yields the Molecular Assembly (MA) index, a causally informed measure of structural complexity. A key insight of the theory is that objects with high MA values
are statistically improbable to form abiotically in high abundance; thus, their presence suggests the involvement of a selective process, such as life. This makes MA a promising candidate for identifying biosignatures and exploring the organizational principles of biochemical systems. This framework aligns well with theories of metabolic evolution, which describe how life's chemical machinery has grown in complexity over time. For instance, the Shell Hypothesis proposes that metabolism evolved in nested, hierarchical layers, with ancient, simple core pathways being supplemented by progressively more complex ones (Scossa and Fernie, 2020). Similarly, the Patchwork Hypothesis suggests that specialized pathways arose from the refinement of primitive, multifunctional enzymes (Lazcano et al., 1995; Fondi et al., 2009; Scossa and Fernie, 2020). These models of stepwise innovation offer a compelling biological context for applying a constructive complexity measure, such as MA. This study, therefore, undertakes a multi-scale investigation using the KEGG database to assess whether MA can serve as a biologically meaningful lens to view the structure and evolution of life's chemistry.

## Materials and methods

In our research, we explored a multi-scale, data-driven methodology to validate the MA index as a biologically relevant complexity metric. The foundation of this work was a comprehensive dataset of over 7,500 metabolites sourced from the Kyoto Encyclopedia of Genes and Genomes (KEGG) database (Kanehisa et al., 2025). For each compound, the MA value was calculated using the assembly-go algorithm, with a 30 -second timeout per molecule to ensure efficient approximation of structural complexity. This compound-level data was then integrated into higher-level contexts, including metabolic pathways classified by evolutionary origin and the complete metabolic networks of 101 representative organisms selected to span the tree of life. R-group compounds and glycan-related pathways were excluded to maintain analytical consistency.

The analysis was conducted across four biological scales to identify functional and evolutionary patterns. At the com-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-097.jpg?height=540&width=762&top_left_y=278&top_left_x=191)
Figure 1: Distributions of absolute differences in MA for compound pairs, grouped by enzyme class.

pound and reaction levels, we correlated MA with standard molecular descriptors and analyzed shifts in complexity across different enzyme classes. A novel similarity metric based on joint MA ( $\mathrm{NCD}_{\mathrm{MA}}$ ) was also developed to quantify shared assembly pathways between molecules, which we validated by clustering the 20 amino acids. At the pathway and organism levels, we investigated large-scale taxonomic signatures by comparing the MA distributions of entire metabolic repertoires using the Wasserstein distance. The resulting distance matrices served as input for dimensionality reduction (via UMAP), network visualization (via Minimum Spanning Tree), and hierarchical clustering to determine whether MA profiles could reconstruct known biological classifications.

## Results

Our work indicated that the MA index captures biologically significant patterns across all scales of biochemical organization, from individual molecules to the tree of life.

## Compound and Reaction-Level Complexity

At the molecular level, MA correlates strongly with features like molecular weight, but also captures nuanced structural information that allows for meaningful similarity clustering of molecules like amino acids. At the reaction level, the change in complexity is predictable and reflects enzyme function. As shown in Figure 1, enzymes like isomerases (class 5) cause almost no change in MA, while enzymes that join or break molecules, like ligases (class 6) and transferases (class 2), cause significant shifts in molecular complexity.

## Pathway and Evolutionary Signatures

MA profiles successfully distinguish metabolic pathways based on their function and evolutionary history. Core
metabolic pathways essential to all life-such as carbohydrate, energy, and amino acid metabolism-exhibit significantly lower median MA values than more specialized and evolutionarily recent pathways like lipid, nucleotide, and cofactor \& vitamin metabolism. This finding supports evolutionary models, like the shell hypothesis, which posit that metabolism evolved in layers of increasing complexity.

## Organism-Level Taxonomic Clustering

The MA distributions of entire metabolic repertoires contain strong taxonomic signals. By comparing the MA profiles of 101 representative organisms using the Wasserstein distance, a clear separation between prokaryotes and eukaryotes emerged. Distinct and well-defined clusters corresponding to animals and plants were also resolved. Organisms that appeared as outliers in the analysis were often parasites or extremophiles, suggesting that MA profiles also reflect metabolic adaptations to unique lifestyles and environments.

## Acknowledgements

This work was executed by Jeppe Van den Langenbergh during his Master's thesis, supervised by Michiel Stock and Bernard De Baets and guided by Bram Spanoghe and Carlos Vigil-Vásquez.

## References

Fondi, M., Emiliani, G., and Fani, R. (2009). Origin and evolution of operons and metabolic pathways. Research in Microbiology, 160(7):502-512.

Jirasek, M., Sharma, A., Bame, J. R., Mehr, S. H. M., Bell, N., Marshall, S. M., Mathis, C., MacLeod, A., Cooper, G. J. T., Swart, M., Mollfulleda, R., and Cronin, L. (2024). Investigating and Quantifying Molecular Complexity Using Assembly Theory and Spectroscopy. ACS Central Science, 10(5):10541064. Publisher: American Chemical Society.

Kanehisa, M., Furumichi, M., Sato, Y., Matsuura, Y., and IshiguroWatanabe, M. (2025). KEGG: biological systems database as a model of the real world. Nucleic Acids Research, 53(D1):D672-D677.

Lazcano, A., Díaz-Villagómez, E., Mills, T., and Oró, J. (1995). On the levels of enzymatic substrate specificity: Implications for the early evolution of metabolic pathways. Advances in Space Research, 15(3):345-356.

Scossa, F. and Fernie, A. R. (2020). The evolution of metabolism: How to test evolutionary hypotheses at the genomic level. Computational and Structural Biotechnology Journal, 18:482-500.

Sharma, A., Czégel, D., Lachmann, M., Kempes, C. P., Walker, S. I., and Cronin, L. (2023). Assembly theory explains and quantifies selection and evolution. Nature, 622(7982):321328. Publisher: Nature Publishing Group.

# Quality-Diversity Search for Open-Ended Machine Learning 

Michiel Stock ${ }^{1}$, Daan Van Hauwermeiren ${ }^{1}$, Bernard De Baets ${ }^{1}$, Steff Taelman ${ }^{1}$, Dries Marzougui ${ }^{2}$, and Maxime Van Haeverbeke ${ }^{1}$<br>${ }^{1}$ KERMIT Research Unit, Department of Data Analysis and Mathematical Modelling, Ghent University, Ghent, Belgium ${ }^{2}$ AI \& Robotics Lab (IDLab-AIRO), Department of Electronics and Information Systems, Ghent University - imec, Ghent, Belgium michiel.stock@ugent.be


#### Abstract

Unlike gradient-based methods, Evolutionary Algorithms use populations and randomness to find novel and performant solutions. Quality-Diversity (QD) algorithms have recently emerged as a distinct paradigm designed to cultivate populations of simultaneously high-performing yet behaviorally diverse solutions. Despite their proven effectiveness and growing popularity, Quality-Diversity methods remain relatively underrecognized and underutilized in the broader data science landscape. We aim to bridge this gap by exploring how QD algorithms can be used to make machine learning more robust and open-ended.


## Submission type: Late Breaking Abstract

## Background

Optimization is a cornerstone of contemporary machine learning, with gradient-based optimization, powered by automatic differentiation frameworks, dominating the landscape. Our work advocates for maintaining diversity throughout the optimization processes at various stages of the data-to-decision pipeline through a Quality-Diversity (QD) search (Figure 1). For instance, when searching for models that fit the training data well, a collection of candidates that differ in terms of the number of parameters, structure (e.g., linear vs. nonlinear), features used, and so on, can explore the search space much more systematically. This systematic exploration can not only lead to finding a better final solution model, but the archive of diverse solutions is valuable in its own right. The diverse models give insight into which approaches work well and which do not. Some models might perform better in certain regions of the feature space. As such, they might complement each other when combined into ensembles. If the problem domain (e.g., detecting animals in images under low-light conditions rather than high-light conditions) or task (e.g., detecting deer instead of cats in images) shifts, some of the models in the archive could serve as starting points for new models, allowing for continual learning. We outline the value of such an archive of diverse, high-quality solutions in data, training, and inference.

## Robustness and continual learning for machine learning

Adopting a Quality-Diversity paradigm can enhance the robustness of machine learning solutions. While deep learning models achieve superhuman performance, they are often fragile, failing on out-of-distribution data due to shortcut learning (Geirhos et al., 2020). This phenomenon, where models exploit dataset-specific cues instead of generalizing, is analogous to the 'sim-to-real' reality gap in robotics (Mouret, 2020). By seeking a diverse population of high-performing models, QD increases the likelihood of discovering robust, intended solutions among the many brittle shortcut solutions.

## Opportunities, challenges and frontiers

Converging trends in modern AI amplify the relevance of the QD paradigm. First, it aligns with Sutton's 'bitter lesson' by providing a general, search-based method that scales with computation. Unlike traditional EAs that converge, QD's divergent search is uniquely suited for massively parallel exploration of open-ended problems (Salimans et al., 2017). Second, QD has a powerful synergy with foundation models; these models can act as sophisticated quality or mutation operators within QD, while QD offers a principled methodology to explore their immense latent spaces and steer them toward genuinely novel and diverse outputs. Third, for agentic AI, QD provides a mechanism to prevent behavioral stagnation and generate a diverse repertoire of strategies, with the resulting QD map serving as a valuable interface for human-AI interaction (Novikov et al., 2025).

Despite its promise, QD faces significant hurdles. A primary challenge is behavioral descriptor engineering, as defining meaningful descriptors remains largely an art, susceptible to misdirecting the search (Stanley and Lehman, 2015). Another is scalability, as traditional grid-based methods scale poorly with the number of behavioral dimensions. While recent advancements, such as learned descriptors (Nickerson and Hu, 2021) and surrogate-assisted search (Kent et al., 2025), show promise, further breakthroughs are still needed.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-099.jpg?height=796&width=1767&top_left_y=276&top_left_x=148)
Figure 1: Overview of Quality-Diversity methods.

Looking ahead, the QD paradigm appears poised to drive progress towards more ambitious frontiers. The diverse, often unexpected solutions uncovered by QD act as crucial 'stepping stones' that open new pathways for innovation (Gaier et al., 2019), a key enabler for achieving openendedness in AI systems (Hughes et al., 2024). This aligns with visions of automating AI discovery itself, where QD's ability to explore vast, combinatorial search spaces makes it a promising approach for the automated invention of new learning algorithms and architectures, moving beyond human intuition alone (Clune, 2019; Real et al., 2020).

## Acknowledgements

This is an extended abstract based on the manuscript "Quality-Diversity Methods for the Modern Data Scientist", which is currently under review.

## References

Clune, J. (2019). AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence. arXiv preprint arXiv:1905.10985.

Gaier, A., Asteroth, A., and Mouret, J.-B. (2019). Are quality diversity algorithms better at generating stepping stones than objective-based search? In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 115-116.

Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665-673.

Hughes, E., Dennis, M. D., Parker-Holder, J., Behbahani, F., Mavalankar, A., Shi, Y., Schaul, T., and Rocktäschel, T.
(2024). Position: Open-endedness is essential for artificial superhuman intelligence. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F., editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 2059720616. PMLR.

Kent, P., Gaier, A., Mouret, J.-B., and Branke, J. (2025). Bayesian optimization for quality diversity search with coupled descriptor functions. IEEE Transactions on Evolutionary Computation, 29(2):302-316.

Mouret, J. B. (2020). Evolving the behavior of machines: from micro to macroevolution. iScience, 23(11):101731.

Nickerson, K. and Hu, T. (2021). Principled quality diversity for ensemble classifiers using MAP-Elites. Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 259-260.

Novikov, A., Vũ, N., Eisenberger, M., Dupont, E., Huang, P.S., Wagner, A. Z., Shirobokov, S., Kozlovskii, B., Ruiz, F. J., Mehrabian, A., et al. (2025). AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131.

Real, E., Liang, C., So, D., and Le, Q. (2020). AutoML-Zero: Evolving machine learning algorithms from scratch. In International Conference on Machine Learning, pages 80078019. PMLR.

Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. (2017). Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864.

Stanley, K. O. and Lehman, J. (2015). Why Greatness Cannot Be Planned: The Myth of the Objective. Springer Cham, 1 edition.

# Architecture Design in Boolean Reservoir Computing 

Christopher Michael Skeide Vibe ${ }^{1}$, Mikkel E. Lepperød ${ }^{2}$, Solve Sæbo ${ }^{3}$, and Stefano Nichele ${ }^{1}$<br>${ }^{1}$ Østold Univeristy College, Norway<br>${ }^{2}$ Norwegian University of Life Sciences, Norway<br>${ }^{3}$ Simula Research Laboratory, Norway<br>christopher.vibe@hiof.no

## Submission type: Late breaking abstract

## Introduction

Artificial life research provides a framework for studying complex systems and emergent behavior through computational modeling. Reservoir Computing (RC) is a flexible, decentralized computation paradigm that projects inputs into a high-dimensional space, where simple rules can solve non-linear tasks (Jaeger, 2010). RC typically has three layers: input (untrained), reservoir (untrained), and output (trained). In this work, we focus on reservoirs made of Random Boolean Networks (RBN) (Gershenson, 2004) and the associated design choices. While we use an abstract graph representation, RC can also be implemented physically, as in neuromorphic computing (Tanaka et al., 2019).

We study RBN reservoirs, using both heterogeneous and homogeneous networks, to solve temporal parity and density tasks that demand increased processing and memory, respectively. Snyder et al. (2012)offers valuable insights exploring aspects such as optimal connectivity, but leaves key details, such as network state initialization and perturbation methods, underexplored. Such aspects are investigated in our study.

## Method

To ensure that the computational performance in the output layer is only a reflection of the reservoir layer, as opposed a suboptimal input layer , the architecture has been divided up into a series of bipartite mappings. All $a \rightarrow b$ below denote a bipartite mapping between the system layers, as depicted in Figure 1a. B denotes a binary data string, such as 110101, hence referred to as a sample.

- B → I Each input bit from the sample is mapped to its own random input node in a $1: 1$ mapping. There are two ways to perturb an input node: override the value, or xor with the existing value. The I node updates its state based on a Lookup Table (LUT) as shown in Figure 1a and 1b.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-100.jpg?height=784&width=858&top_left_y=813&top_left_x=1064)
Figure 1: Highlevel setup

The LUT update has no effect in override mode but does in xor mode.

- I → I There are no edges from input to input.
- $\mathrm{I} \rightarrow \mathrm{R}$ Each input node in $I \rightarrow R$ can have a variable number of outgoing edges, described by a bipartite shorthand such as out-a:b:1/b: each input node has $a$ guaranteed connections and up to $b-1$ additional edges, each with probability $1 / b$. If the probability is 1 , the mode is deterministic; otherwise, it is stochastic.
- R→I There are no edges from reservoir to input.
- $\mathrm{R} \rightarrow \mathrm{R}$ The RBN itself represents the reservoir, which serves as the core computational element, comprising of $R$ nodes. These nodes are connected by edges, with the average number of edges per node defined as $k_{\text {avg }}=\frac{\Sigma K_{i}}{R}$, where $K_{i}$ are the incoming edges per node. The average connectivity will be varied from integers [1-10] to find
optimal compute conditions and investigate how design relationships change. The construction of these reservoirs can occur in two different structural formats: homogeneous and heterogeneous as defined below.

Reservoir networks are categorized into two distinct configurations based on their connectivity patterns. Homogeneous reservoirs feature all nodes with $k_{i}=c$ incoming edges, creating an even distribution of connections across the entire network. This uniformity allows for predictable connectivity making differentiation in interaction patterns more dependent on the LUT and input layer. In contrast, heterogeneous reservoirs feature nodes with varying in-degrees, resulting in an uneven distribution of connections. Each node's connections are randomly sampled from a constrained discrete uniform distribution with $k_{i}=\left[0: k_{\text {max }}\right]$, promoting diversity within the network connectivity, potentially enhancing its computational capacity through varied interaction patterns.

Before each batch of 100 samples, the reservoir is reset to an initial state. The reservoir is instantiated as zeros: All nodes set to state 0 and random: Each node has a random state ( $p=0.5$ ).

- Output Layer (O): The linear readout from the reservoir layer is dense (all of R).


## Task-Agnostic Performance

Kernel Quality (KQ) and Generalization Rank (GR) are two metrics that can predict performance without the need for a specific task (Jensen and Tufte, 2020). These metrics evaluate the variation in reservoir dynamics by examining the rank of a history matrix. This approach provides insight into the system's ability to differentiate between inputs and generalize states that have similar inputs.

## Selected Results and Summary

Performaces are evaluated on three benchmarks: parity, density, and 2D path integration. A subset of preliminary visual results are in Figure 2 and Table 2c. Further experiments vary the task, delay, and window. The window selects I relevant bits, while the delay shifts this window, introducing noise bits around them. For instance, a window of 5 with a delay of 1 results in 5 noise bits: 1 before and 4 after, assuming 10 total bits. Figures 2a and 2b where selected to show network connectivity $R_{\_} k_{\_} a v g$ being both harmful and beneficial to success rate, defined in Figure 2.

The exploration uncovers key practical design choices when using RBN for RC. By adopting both homogeneous and heterogeneous designs, the study describes how these variations influence the processing capabilities at the core reservoir layer. By experimenting with input perturbation methods focusing on deterministic and stochastic degrees of connectivity, the computation capability can be assessed from a perspective with and without the input layer. Full results will be presented as a poster.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-101.jpg?height=2099&width=682&top_left_y=264&top_left_x=1153)
Figure 2: Radial values signify average success rate of a design with a threshold of $90 \%$ accuracy. $R_{-} k_{-} a v g$ contours indicate the reservoirs average connectivity. The table rows are ordered by a gray code to minimize difference between directions and columns ordered by success rate variance.

## References

Gershenson, C. (2004). Introduction to Random Boolean Networks. arXiv:nlin/0408006.

Jaeger, H. (2010). The "echo state" approach to analysing and training recurrent neural networks - with an Erratum note.

Jensen, J. H. and Tufte, G. (2020). Reservoir Computing in Artificial Spin Ice. In ALIFE 2020: The 2020 Conference on Artificial Life, pages 376-383. MIT Press.

Snyder, D., Goudarzi, A., and Teuscher, C. (2012). Finding Optimal Random Boolean Networks for Reservoir Computing. In ALIFE 2012: The Thirteenth International Conference on the Synthesis and Simulation of Living Systems, pages 259-266.

Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., and Hirose, A. (2019). Recent advances in physical reservoir computing: A review. Neural Networks, 115:100-123.

# Language Cellular Automata: Guiding Morphogenesis with Natural Language 

Elias Najarro ${ }^{1}$, Eleni Nisioti, ${ }^{1}$, Milton L. Montero, ${ }^{1}$ Sebastian Risi, ${ }^{1,2}$<br>${ }^{1}$ IT University of Copenhagen, Denmark; ${ }^{2}$ Sakana AI; Japan


#### Abstract

Self-organising systems, such as morphogenesis, produce complex global patterns from simple local interactions, yet guiding these processes in a controlled way remains challenging. Cellular automata (CA) have long been used as abstract models of such dynamics, and neural cellular automata (NCA) extend them by learning update rules from data. In this work, we introduce Language Cellular Automata (LCA), a self-organising model in which cell states are conditioned with natural language by injecting text embedding into each individual cell, allowing global patterns to emerge that match the text prompt. Our experiments show that LCA can reliably assemble into target shapes described in natural language, and generalise to unseen combinations, such as novel color-shape pairings. The results demonstrate that language can serve as a powerful control signal for self-organising processes, opening new possibilities for guiding emergent dynamics in artificial and natural systems.


## Submission type: Late Breaking Abstract

## Introduction

Self-organising phenomena represent a cornerstone in the study of complex systems. They manifest as intricate, and often unpredictable, behaviours that arise from simple interactions among components of a system. These phenomena-central to disciplines ranging from biology to physics-challenge our understanding and modelling capabilities, especially in regard to the question of how to steer them in a controlled manner.

Cellular growth and morphogenesis are examples of selforganising phenomena where the interplay of expressed proteins and the environmental conditions encode the intended tissue morphology, and eventually, the whole organism structure. A classical computational model of morphogenesis are Cellular automata (CA). CA are models inspired themselves by biological cellular growth that display complex dynamical patterns emerging purely through local interaction of its elements-typically, via simple rules. Recently, neural cellular automata (NCA), which replace the update rule of the CA by a neural network Wulff and Hertz (1992);

Tavares et al. (2006), have gained popularity by taking advantage of modern deep learning tools. Traditionally, CA have been used to model phenomena in a bottom-up way, however, recent research has demonstrated the potential of NCA to replicate both static and dynamic patterns Mordvintsev et al. (2020); Sudhakaran et al. (2021); Richardson et al. (2023), opening the door to a top-down modelling approach to self-organising phenomena.

## Language Cellular Automata

Recently, natural language has become a popular interface to machine learning models: from text assistants to multimodal generative AI. In this work, we demonstrate how selforganising processes can also be guided with natural language. Specifically, we show that a neural CA model of morphogenesis can be prompted with a text, resulting in the CA self-organising into a pattern that match the text description. Fig. 1 shows the architecture of Language Cellular Automata (LCA) model.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-104.jpg?height=273&width=805&top_left_y=1681&top_left_x=1087)
Figure 1: Language Cellular Automata (LCA) architecture. The target shape is represented by 3 visible RGB channels. The transition function of the CA consist of the following steps: first, at time step N , information spread in the CA via learned convolutional filters of dimension $3 \times 3$. The text description is then concatenated to the CA latent state, and finally each cell is updated by a fully connected neural network resulting on the update CA state N +1 .

We curated a dataset consisting of 720 pairs of patterns and their corresponding text descriptions; a subset of the dataset is shown in Fig. 2. The natural language descriptions are mapped onto an embedding space using a pre-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-105.jpg?height=508&width=791&top_left_y=274&top_left_x=178)
Figure 2: Compositional dataset. Subset of the full 720 image-text dataset. The LCA is trained on a dataset composed of animal-emojis with different colors. By holding out of the training dataset some combinations, we can evaluate the capacity of the LCA of composing visual features when conditioned with prompts not seen during training.

trained embedding model Mikolov et al. (2013); Pennington et al. (2014); Reimers and Gurevych (2019); Radford et al. (2021). In this semantic vector space, two sentences with similar meaning live nearby-e.g. a green lizard and an orange lizard should have similar representations. Our experiments show that the LCA model performs well with different embedding models. Like with any CA model, each cell in the LCA runs the exact same program, and contains an identical copy of the target pattern (e.g., a blue lizard) in the form of a vector embedding-analogously to how biological cells contain the same DNA copy acting as a blueprint of the organism's morphology. The global pattern unfold over time through the cells interactions. See Fig. 1 for an overview of the model architecture.

Our results show that the LCA can indeed learn to selforganise into different 2D morphologies when prompted with natural language. The trained model is capable of selfassembling onto different target patterns with different attributes; Fig. 3 shows a subset of the patterns generated by a single LCA prompted with previously unseen text descriptions.

## Conclusion and Future work

While in this work we focus on guiding the CA behaviour with natural language, embedding models can be used to generate numerical representations for many modalities of data, e.g. omic data in the context of biological morphogenesis or cell differentiation, or weather patterns in climate modelling. The LCA architecture is agnostic to the semantics of the data used to guide it, provided that: $\mathbf{a}$. the system can be represented a set of locally interacting agents-be it spatially embedded cells in a grid, or nodes in a network, b. the space of target states is sufficiently large and dense,

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-105.jpg?height=1041&width=704&top_left_y=278&top_left_x=1135)
Figure 3: Compositional generalization. Patterns generated from a single LCA prompted on unseen color-animal combinations-e.g., a pink squirrel, purple worm, etc. The capacity of the LCA to generate coherent patterns, where the general morphology of the animal is correctly reconstructed but the color follows the new prompt, suggests that the model has learned to meaningfully interpolate in the text embedding space.

c. there exist an embedding model that meaningfully encodes target macro states of the system, and d. enough data is available, or can be synthetically generated.

In summary, this work illustrates the potential synergies of bringing together machine learning techniques and selforganising models. It outlines the very early stages of a roadmap to control self-organising system, such as biological morphogenesis, via natural language, or via other embedding-like guiding signal.

## Acknowledgements

This project was supported by the European Research Council (ERC) grant no. 101045094, project GROW$A I$ and by the Novo Nordisk Foundation, grant number NNF23OC0086722.

## References

Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J. (2013). Distributed representations of words and phrases
and their compositionality. Advances in neural information processing systems, 26 .

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing neural cellular automata. Distill.

Pennington, J., Socher, R., and Manning, C. D. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532-1543.

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. (2021). Learning Transferable Visual Models From Natural Language Supervision. arXiv.

Reimers, N. and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv.

Richardson, A. D., Antal, T., Blythe, R. A., and Schumacher, L. J. (2023). Learning spatio-temporal patterns with Neural Cellular Automata. arXiv.

Sudhakaran, S., Grbic, D., Li, S., Katona, A., Najarro, E., Glanois, C., and Risi, S. (2021). Growing 3D Artefacts and Functional Machines with Neural Cellular Automata. arXiv.

Tavares, J., Kreutzer, C., and Fedor, A. (2006). Neuro-cellular automata: Connecting cellular automata, neural networks and evolution.

Wulff, N. and Hertz, J. A. (1992). Learning cellular automaton dynamics with neural networks. Advances in Neural Information Processing Systems, 5:631-638.

# Visualizing the Structure of Lenia Parameter Space 

Barbora Hudcová ${ }^{1 *}$, František Dušek ${ }^{2}$, Marco Tuccio ${ }^{3}$, and Clément Hongler ${ }^{1}$<br>${ }^{1}$ EPFL, Switzerland, *barbora.hudcova@epfl.ch<br>${ }^{2}$ CIIRC, CTU and FIT, CTU, Czech Republic<br>${ }^{3}$ Universitat Pompeu Fabra, Spain


#### Abstract

Continuous cellular automata are rocketing in popularity, yet developing a theoretical understanding of their behaviour remains a challenge. In the case of Lenia, a few fundamental open problems include determining what exactly constitutes a soliton, what is the overall structure of the parameter space, and where do the solitons occur in it. In this abstract, we present a new method to automatically classify Lenia systems into four qualitatively different dynamical classes. This allows us to detect moving solitons, and to provide an interactive visualization of Lenia's parameter space structure on our website https://lenia-explorer.vercel. app /. The results shed new light on the above-mentioned questions and lead to several observations: the existence of new soliton families for parameters where they were not believed to exist, or the universality of the phase space structure across various kernels.


## Submission type: Late Breaking Abstract

website: https://lenia-explorer.vercel.app/ data: https://bit.ly/lenia-explorer-data code: https://bit.ly/lenia-explorer-code

## Introduction

Lenia has recently become one of the most popular Alife models due to the beautiful patterns that frequently emerge in its dynamics Chan (2018). Despite the profound interest Kawaguchi et al. (2021); Hamon et al. (2022); Plantec et al. (2023); Papadopoulos and Guichard (2025), our theoretical understanding of Lenia remains very limited Yevenko et al. (2025); Kojima et al. (2025); we list a few open questions:

- What constitutes solitons, and how to automatically detect them?
- Where in the Lenia parameter space do solitons emerge?
- How does the emergence of solitons depend on the choice of initial configurations?
- What is the structure of the Lenia parameter space and how does the shape of the kernel affect it?

In this work we distinguish four qualitatively different types of Lenia's dynamical behaviour and we present a new method to automatically classify each system into one of
them. This allows us to visualize the "phase space" for various Lenia kernels. The results can be interactively explored at our website, which helps to build an important understanding to the above mentioned questions, and which identifies soliton regions for a variety of kernels.

In Yevenko (2024) the Lenia parameter space was shown to exhibit a fractal structure. The most thorough identification of solitons was done manually in Chan (2018), Figure 9 for one fixed kernel.

## Method

We study the classical variant of Lenia with a single channel; a kernel $K: \mathbb{R} \rightarrow[0,1]$, and a growth function $G:[0,1] \rightarrow [-1,1]$. An $n \times n$ configuration at time $t$ is $A^{t} \in[0,1]^{n \times n}$; the Lenia update with a time-step $\Delta t \in \mathbb{R}_{\geq 0}$ at position $x$ is given by

$$
A^{t+\Delta t}(x)=\left[A^{t}+\Delta t G\left(K * A^{t}(x)\right)\right]_{0}^{1}
$$

where $*$ denotes the convolution operation; for details see Chan (2018). In this work, we fix $\Delta t=0.1$ and consider $G$ to be of the form $G_{\mu, \sigma}(x)=2 e^{-\frac{(x-\mu)^{2}}{2 \sigma^{2}}}-1 ; \mu, \sigma \in (0,1)$. Thus, for a fixed kernel $K$, each Lenia system is characterized by the values $\mu$ and $\sigma$. Below, we define the four dynamical classes and our classification method.

## Dynamical Phases of Lenia Systems

We fix a system with kernel $K$ and growth function parameters $\mu$ and $\sigma$. First, we define the dynamical phase of a given initial configuration. Then, we traverse the space of initial configurations to determine the system's overall behaviour.

Classifying Initial Configurations For $A^{0} \in[0,1]^{n \times n}$, we compute the trajectory $A^{0}, A^{\Delta t}, A^{2 \Delta t}, \ldots, A^{T_{\max } \Delta t}$ with $T_{\text {max }} \approx 7000$ and assign to $A^{0}$ one of the following phases ${ }^{1}$ :
→ Stable Phase: The trajectory enters a loop.

- Metastable Phase: The trajectory does not enter a loop, but its center of mass stabilizes around its long-term mean.
- Unclassified: Neither of the two cases above.

A typical final configuration $A^{T_{\text {max }}}$ of each phase is shown in Figure 1.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-109.jpg?height=232&width=643&top_left_y=264&top_left_x=248)
Figure 1: a) All activity dies out. b) Activity expands to the whole array. c) All moving solitons get "unclassified".

Crucially, after analyzing over $10^{5}$ Lenia systems, we observed a strong correspondence between unclassified initial configurations and the emergence of moving solitons.

Dynamical Classes of Lenia Systems We now address a key question: how to choose initial configurations. The dynamics of many CAs depends strongly on this choice Behrens et al. (2024), and traversing Lenia's initialconfiguration space is essential to capture its full diversity. Following Chan (2018), we initialize with a noise patch (uniform random values in $[0,1]$ ) surrounded by zeros. To avoid bias toward specific shapes, we use the maximum-entropy principle: the shape of the patches is defined by random Voronoi polygons ${ }^{1}$. We traverse the configuration space by varying the size of noise patches; for a $100 \times 100$ grid, polygons of areas $10^{2}, 20^{2}, \ldots, 90^{2}$ are used. For each, we generate 64 configurations and determine their phases (Fig. 2).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-109.jpg?height=296&width=839&top_left_y=1338&top_left_x=155)
Figure 2: Traversing the space of initial configurations by increasing the area of noise in the shape of random polygons.

We can summarize the matrix data of Fig. 2 in a simple plot which, for each polygon area, shows the proportion of initial configurations classified into each phase. After analyzing a variety of systems, we observed four typical classes of dynamical behaviour summarized in Figure 3.

To summarize the method, we described an algorithm that assigns to each Lenia system one of four classes of dynamical behaviour, allowing us to automatically detect systems (and initial configuration regions) where solitons occur.

## Results

We have analyzed 8 kernels of various shapes. For each kernel, we vary the growth function parameters $\mu$ and $\sigma$, obtaining a $\mu-\sigma$ plane of around 10000 systems, assigning a dynamical class to each. The complete results are made available on our interactive website ${ }^{2}$ and we encourage the

[^5]![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-109.jpg?height=206&width=759&top_left_y=283&top_left_x=1096)
Figure 3: $x$-axis: polygon sizes, $y$-axis: proportion of configurations in each phase. a) All configurations enter the stable phase. b) All configurations enter the metastable phase. c) A transition from stable to metastable phase as the patches of noise increase in size. d) A transition with solitons occuring around the transition region.

reader to explore the fascinating range of emerging solitons. We illustrate the trend for the kernel $K(r)=e^{4-\frac{1}{r(1-r)}}$ with radius $R=13$ in Fig. 4 .

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-109.jpg?height=337&width=791&top_left_y=982&top_left_x=1101)
Figure 4: (Left) "Phase space" of Lenia's dynamical classes for a fixed kernel while varying $\mu$ and $\sigma$. Dark orange region contains systems with emerging solitons, some of them showcased in the (middle). (Right) Analogous phase spaces for various kernel shapes depicted next to them with $0.1 \leq \mu \leq 0.5$ and $0.0<\sigma \leq 0.1$.
In Chan (2018), Fig. 9, the same $\mu-\sigma$ plane was studied

for an analogous kernel, though only for $\mu \leq 0.5 \sigma \leq 0.12$ as no solitons were believed to exist for larger values. Figure 4 thus shows the discovery of a new soliton families such as b) and c).

## Conclusion

The dynamical classification of Lenia allowed us to make the following observations:

- The method enabled automatic detection of solitons.
- Solitons seem to occur in a very specific region of initial configurations, which typically lies in a transition region between the stable and metastable phase.
- We discovered new soliton families, such as Figure 4 c).
- We observe that a variety of kernels give rise to a qualitatively similar $\mu-\sigma$ "phase space", strinkingly resembling the phase transition of water.
We believe that our results could help to give an analytical description between the stable and transition phase (more numerical results in Papadopoulos et al. (2024)) as well as between the transition and metastable phase. Analytically characterizing the soliton region seems a more challenging, yet a very interesting goal.


## Acknowledgements

We would like to thank Bert Chan, Vassilis Papadopoulos, Ehsan Pajouheshgar, Eugène Bergeron, Franck Gabriel, and João Penedones for inspiring discussions. This work was funded by NCCR SwissMAP.

## References

Behrens, F., Hudcová, B., and Zdeborová, L. (2024). Dynamical phase transitions in graph cellular automata. Physical Review $E$, 109(4):044312.

Chan, B. W.-C. (2018). Lenia-biology of artificial life. arXiv preprint arXiv:1812.05433.

Hamon, G., Etcheverry, M., Chan, B. W.-C., Moulin-Frier, C., and Oudeyer, P.-Y. (2022). Learning sensorimotor agency in cellular automata.

Kawaguchi, T., Suzuki, R., Arita, T., and Chan, B. (2021). Introducing asymptotics to the state-updating rule in lenia. In Artificial Life Conference Proceedings 33, volume 2021, page 91. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info ....

Kojima, H., Yevenko, I., and Ikegami, T. (2025). The glider equation for asymptotic lenia. arXiv preprint arXiv:2508.04167.

Papadopoulos, V., Doat, G., Renard, A., and Hongler, C. (2024). Looking for complexity at phase boundaries in continuous cellular automata. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 179182.

Papadopoulos, V. and Guichard, E. (2025). Mace: General mass conserving dynamics for cellular automata.

Plantec, E., Hamon, G., Etcheverry, M., Oudeyer, P.-Y., MoulinFrier, C., and Chan, B. W.-C. (2023). Flow-lenia: Towards open-ended evolution in cellular automata through mass conservation and parameter localization. In Artificial Life Conference Proceedings 35, volume 2023, page 131. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info . . . .

Yevenko, I. (2024). Classifying the fractal parameter space of the lenia orbium. In Artificial Life Conference Proceedings 36, volume 2024, page 14. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info....

Yevenko, I., Kojima, H., and Nehaniv, C. L. (2025). Using dynamical systems theory to quantify complexity in asymptotic lenia. arXiv preprint arXiv:2508.02935.

# Evolvable Chemotons: Toward the Integration of Autonomy and Evolution 

Kuzuya Horibe ${ }^{1, *}$, Daichi G. Suzuki ${ }^{2,3, *}$<br>${ }^{1}$ RIKEN-CBS, JAPAN<br>${ }^{2}$ University of Tsukuba, JAPAN<br>${ }^{3}$ Corresponding author: suzuki.daichi.gp@u.tsukuba.ac.jp<br>* These authors contributed equally to this work.


#### Abstract

In this study, we provide a relatively simple simulation framework for constructing artificial life (ALife) with both autonomous and evolutionary aspects by extending Gánti's chemoton model. While the original chemoton incorporates metabolism, membrane, and genetic templates, it lacks a mechanism for phenotypic variation, preventing true evolutionary dynamics. To address this, we introduced a geno-type-phenotype coupling by linking templates to a second autocatalytic cycle, enabling mutations to affect phenotype and be subject to selection. Using a genetic algorithm, we simulated populations of chemotons over generations. Results showed that chemotons without access to the new cycle remained in a stable but complexity-limited regime, while lineages acquiring the additional metabolic set evolved longer templates. These findings demonstrate that even simple replicator systems can achieve primitive evolvability, highlighting structural thresholds and rare innovations as key drivers. Our framework provides a tractable model for exploring autonomy and evolution in ALife.


## Submission type: Late Breaking Abstracts

## Introduction

It is widely recognized that autonomy and evolution are essential factors for life (Ruiz-Mirazo et al., 2004). In artificial life (ALife) research, however, these two factors have tended to be studied separately, with a few exceptional studies.

One of the most pioneering (and less discussed in the field of ALife) examples is the chemoton model of cells (Gánti, 1975, 2003). A chemoton consists of three subsystems: metabolic cycles, membrane boundaries, and genetic templates. This triadic characterization of life has been widely accepted (Szathmáry, 2005; Rasmussen et al., 2016; Solé, 2009; Nurse, 2020). The first two (metabolism and boundaries) represent the autonomous or autopoietic aspect of life (see Luisi, 2006, for the comparison), while the last (templates) indicates the potential to incorporate genetic inheritance. However, this Gántian model does not fully accommodate the evolutionary aspect of life because it focuses primarily on a single individual chemoton, not a lineage or population.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-112.jpg?height=606&width=556&top_left_y=831&top_left_x=1215)
Figure 1: The "evolvable chemoton" model. A chemoton with two metabolic cycles, based on Fernando and Di Paolo's (2004) model (refer to this paper for abbreviations).

For the integration of the autonomy and evolution, here we offer an extended chemoton model encompassing these two aspects. The classical chemoton offers autopoietic subsystems (i.e., the metabolism, membrane, and template), but its template lacks phenotypic effect, so genetic changes cannot generate novelties subject to selection. To overcome this disadvantage, we add a genotype-phenotype coupling by letting templates have access to a second autocatalytic loop, and we examined the populational dynamics of these chemotons through simulational evolution.

## Methods

We first prepared an initial population with $P=10$ chemotons only with a gene set for metabolism $\mathrm{A}\left(M_{\mathrm{A}}\right)$, and then ran the genetic algorithm with the following growth, selection, and mutagenesis phases for 50 generations. The biochemical reactions for metabolism, membrane growth, and template replication were described by ordinary differential equation (ODE) system, acccording to previous studies (Csendes, 1984; Fernando and Di Paolo, 2004). All
initial chemotons were set to have $N \approx 25$ templates under the normal distribution. Growth phase: ODEs were solved through LSODA until 1.0 simulation time, during which a default chemoton can divide 9 times; chemotons that reached a surface-area threshold were allowed to divide into two daughter chemotons, copying their $N$ and the gene set status coded in their templates. Selection phase: the grown population was randomly selected to $P$ individualsthus trajectory that divided more had a chance to be selected. Mutagenesis phase: every survivor or selected chemoton received a Gaussian perturbation ( $\sigma=2$ ) to $N$ for mutation. If a chemoton obtained a new template with $N \geq 40$ and it lacked the gene set for metabolism $\mathrm{B}\left(M_{\mathrm{B}}\right)$, it was allowed to acquire the gene set for metabolism B with a probability $p=0.3$. If the chemoton with gene sets for both metabolism A and B obtained a new template with $N$ below 40, the chemoton was forced to lose the gene set for metabolism B. Chemotons with $N<20$ templates were discarded as lethal.

All state variables were initialized as follows. The series of metabolite intermediates $\left[A_{1}\right]-\left[A_{5}\right]$ began at $[1.0,1.8,1.9$, 1.7, 10.0], and $\left[B_{1}\right]-\left[B_{5}\right]$ were all zero (inactive). Intracellular monomer concentration $V$ started at 40.0, and the initial template abundance $p V_{0}$ was set to 0.01 . The residue pool $R$ was 0.5 . Membrane precursor states $T_{P}, T_{S}$, and $T$ were initialized to 17,14 , and 0 , respectively. Surface-area and volume proxy variables $S$ and $Q$ both began at 1.0. The smoothing variable tmpl_len was set equal to $N$, and the Boolean metabolism-B flag has_met_B was initialized to false.

## Results

As shown in Figure 2a, the B-lacking trajectory keep $N$ oscillating between 22-32, never breaching the gate $N=40$. In contrast, B-acquired trajectory shows a pivotal event: once any cell surpasses $N=40$ and wins thdraw of probability $p$ (In this simulation, $p=0.3$ ), B carriers divide faster, pushing the mean $N$ into the 40-50 zone and drithe prevalence of B to $100 \%$ in 5 generations (Figure 2b).

These results highlight that evolutionary success requires both structural preconditions (such as sufficiently long templates) and rare beneficial innovations (exemplified by metabolism B). Remarkably, even under realistic mutational and probabilistic constraints, adaptive evolution remains accessible in a substantial fraction of runs. To conclude, our evolvable chemoton model demonstrates that simple replicator systems can robustly evolve toward greater complexity and productivity.

## Discussion

The simulation results shown here suggest that our model manifests a primitive sort of evolvability and adaptation, as evolution is typically defined as temporal changes in the genetic frequency within a gene pool; although shorter tem-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-113.jpg?height=931&width=840&top_left_y=278&top_left_x=1073)
Figure 2: Outcomes for the evolvable chemoton model. (a) B-lacking and (b) B-acquired trajectories. For each trajectory, the mean template length N with range (above; blue lines with light blue bands) and the prevalence of metabolism B in the population (below) are shown. Horizontal red and green dashed lines mark the survival threshold ( $\mathrm{N}=20$ ) and the acquisition threshold for metabolism B ( $\mathrm{N}=40$ ), respectively.

plates facilitate a high cell division rate, some populations acquire new gene sets for an additional metabolic cycle by retaining longer templates. In addition, our results well illustrate the genomic evolution of organisms, as it is suggested that the increase of genome size is accompanied by the increase of organismal complexity (Sharov, 2006).

With the present model, we provide a relatively simple simulation framework for constructing ALife with both autonomous and evolutionary aspects. This model can be used for various advanced analyses. For example, it should be worth to examine the responce to environmental changes of nutrient supply. Also, our model can be sophisticated to embody a more realistic coding system. One way for this to be achieved is by arranging a series of genes that code for respective enzymes involved in the biochemical reactions in the chemoton model.

## Acknowledgements

This work is supported by JSPS Grants-in-Aid for Scientific Research (Grant Numbers JP24H01538 for D.G.S. and JP24K20859 for K.H.).

## References

Csendes, T. (1984). A simulation study on the chemoton. Kybernetes, 13(2):79-85.

Fernando, C. and Di Paolo, E. (2004). The chemoton: a model for the origin of long rna templates. In Artificial Life IX: Proceedings of the Ninth International Conference on the Simulation and Synthesis of Life, pages 1-8. MIT Press Cambridge, MA.

Gánti, T. (1975). Organization of chemical reactions into dividing and metabolizing units: the chemotons. BioSystems, 7(1):1521.

Gánti, T. (2003). The Principles of Life. Oxford University Press.
Luisi, P. L. (2006). The emergence of life: from chemical origins to synthetic biology. Cambridge University Press.

Nurse, P. (2020). What Is Life?: understand biology in five steps. Scribe Publications Pty Limited.

Rasmussen, S., Constantinescu, A., and Svaneborg, C. (2016). Generating minimal living systems from non-living materials and increasing their evolutionary abilities. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 371(1701).

Ruiz-Mirazo, K., Peretó, J., and Moreno, A. (2004). A universal definition of life: autonomy and open-ended evolution. Origins of life and evolution of the biosphere: the journal of the International Society for the Study of the Origin of Life, 34(3):323-346.

Sharov, A. A. (2006). Genome increase as a clock for the origin and evolution of life. Biology Direct, 1():17.

Solé, R. V. (2009). Evolution and self-assembly of protocells. The international journal of biochemistry \& cell biology, 41(2):274-284.

Szathmáry, E. (2005). Life: in search of the simplest cell. Nature, 433(7025):469-470.

# Integrative Formulation of Autonomous and Evolving Aspects of Life by Category Theory 

Ryuzo Hirota ${ }^{1, *}$ and Daichi G. Suzuki ${ }^{2,3, *}$<br>${ }^{1}$ University of Tokyo, JAPAN<br>${ }^{2}$ University of Tsukuba, JAPAN<br>${ }^{3}$ Corresponding author: suzuki.daichi.gp@u.tsukuba.ac.jp<br>* These authors contributed equally to this work.


#### Abstract

Autonomy and open-ended evolution are widely acknowledged as fundamental characteristics of life, yet these aspects have largely been studied in isolation within the field of artificial life. In this study, we propose an integrative formulation of these two aspects using category-theory. Here, autonomy is formalized as a monoid of (self-)mediation, while open-ended evolution is modeled through a meta-level free category, capturing the diachronic identity of evolving systems. Our model highlights rich and dynamically changing self-mediation, distinguishing life from non-living systems.


## Submission type: Late Breaking Abstracts

## Introduction

It is widely recognized that autonomy and open-ended evolution (OEE) are essential factors for life (Ruiz-Mirazo et al., 2004). Autonomy refers to a system's capacity to actively regulate and maintain its own identity through interactions with its environment. OEE, on the other hand, denotes the continuous production of novel and increasingly complex forms without a predefined limit (Taylor et al., 2016). These two concepts are considered central to understanding the nature of life, both natural and artificial. In artificial life (ALife) research, however, these two factors have tended to be studied separately, with a few exceptional studies. For example, Witkowski and Ikegami (2019) explores how to evolve the Boid model that spontaneously forms clusters, which can be viewed as a kind of autonomous self-individualization at a higher organizational level, in an open-ended manner (see also Plantec et al., 2023).

For the integration of the autonomy and open-ended evolution aspects, we first need a theoretical framework that can describe both of them. Nevertheless, it is suggested that setbased mathematical theories are not suitable to account for the diachronic evolution of living systems (Kauffman and Roli, 2021). Basically, set-theoretic approaches primarily focus on "elements" and secondarily on changes and transformation. For example, when describing evolution, they assume a "space" (or "landscape") consisting of all possi-
ble traits, and then express evolutionary changes as "trajectories" within that space (see Montévil, 2019), which, as Kauffman and Roli (2021) point out, ultimately fails to capture the open-ended nature of evolution.

In this paper, we instead aim to use category theory for the formulation of the autonomous and evolving aspects. In a category, each object is characterized not by the internal elements or components it has independently of its relationship to others, as in set theory, but by what arrows it has to and from other objects. In other words, category theory focuses on not what it is but what it does or how it behaves in relation to other things. Also, the one-to-one correspondence between an object and its identity arrow allows them to be identified. In other words, what is represented as an object in a category is also a special case of arrows, or "being" as a kind of "doing". From our point of view, this feature is well-suited for describing evolving nature of life because it primarily highlights changes or transformation represented by arrows (Eilenberg and Maclane, 1945), and does not depend on defining the "entire space of possibilities."

## Autonomy and monoid

Hirota et al. (2023) propose a category-theoretic formulation of the autonomy of living systems as a "monoid of mediation". A monoid is a category with only one object, and therefore all of its arrows are self-arrows from the object to itself and also composable with each other through the object. Thus, according to this formulation, autonomy refers to a monoid-like structure in which various loops of selfmediating processes (represented by the self-arrows) connect with each other through a single object, representing the mere "being" or identity of the system, which would be empty if isolated from other (non-trivial) forms of selfmediation.

While this formulation successfully captures the interdependence between self-distinction (i.e., existence as something distinct from one's surroundings) and self-production (i.e., producing one's own components through interaction with the environment), as highlighted by the theory of autopoiesis (Maturana and Varela, 1980), it also seems to re-
flect one of the problems that have been pointed out in the theory: namely, it is necessary but not sufficient for discriminating the living from non-living by itself (Bourgine and Stewart, 2004; Bitbol and Luisi, 2004). For example, phenomena such as candle flames and typhoons, which are called "dissipative structures," can be said to be selfmediating in the sense that they generate dynamics that in turn sustain their own existence (McGregor and Virgo, 2011; Mossio and Moreno, 2010), and therefore can possess the structure as a monoid of mediation.

From this perspective, what essentially distinguishes living systems from non-living self-mediating systems might be the rich variety of the forms of self-mediation and their historical dependence. While flames and typhoons have only single form of self-mediation, living systems, as implied by Bourgine and Stewart (2004), have multiple and everchanging forms of self-mediation, and it would be the orchestration of them that makes living organisms non-trivial.

## OEE and meta-level free category

Recently, Suzuki and Saigo (2025) proposed a processoriented formulation of the homology concept in biology (i.e., the sameness of biological characters) based on category theory. In addition, they suggest that their formulation is also applicable to the sameness of biological individuals (i.e., self-maintanance of their identities).

Here, we simplify their formulation to demonstrate how category theory can integrate the autonomous and the evolutionary aspects of life.

As discussed above, a biological individual organism $A$ can be regarded as a diachronic sustention of a monoid. To illustrate its self-maintanance of identity, we first define the entire biochemical network of $A$ at the timepoint $t$ and $t+1$ as $\mathcal{G}_{A}^{t}$ and $\mathcal{G}_{A}^{(t+1)}$, respectively, where $\mathcal{G}_{A}$ can be regarded as a self-mediating monoid. To make a meta-level directed graph, let us then draw an edge $e: \mathcal{G}_{A}^{t} \longrightarrow \mathcal{G}_{A}^{(t+1)}$ whenever they share at least some part of the network. Furthermore, we have a symmetric graph by adding an edge $e^{*}: \mathcal{G}_{A}^{(t+1)} \longrightarrow \mathcal{G}_{A}^{t}$. By considering $e^{* *}=e$, the operator * means an "involution" on edges. Let us write this symmetric graph as $\Gamma$.

For any graph $\mathcal{Q}=\left(V_{\mathcal{Q}}, E_{\mathcal{Q}}\right)$, where elements of $V_{\mathcal{Q}}$ and $E_{\mathcal{Q}}$ are called vertices and edges of $\mathcal{Q}$, respectively, we can construct a free category $\mathcal{C}(\mathcal{Q})$ of $\mathcal{Q}$, whose objects and arrows are vertices and "paths" (simply put, a coherent sequence of directed edges), respectively. The free category $\mathcal{C}(\mathcal{Q})$ becomes a dagger category (or category with involution) when $\mathcal{Q}$ is symmetric. Specifically, for our meta-level symmetric graph $\Gamma$ equipped with the operation *, $\mathcal{C}(\Gamma)$ becomes a dagger category.

We suggest that this $\mathcal{C}(\Gamma)$ represents the diachronic identity of $\mathcal{G}_{A}$. When we consider a time series of

$$
\mathcal{G}_{A}: \mathcal{G}_{A}^{t}, \mathcal{G}_{A}^{(t+1)}, \ldots, \mathcal{G}_{A}^{(t+n)}
$$

we can construct $\mathcal{C}(\Gamma)$ even if the entire elements of $\mathcal{G}_{A}$ are completely replaced from the initial condition $\mathcal{G}_{A}^{t}$ and the final condition $\mathcal{G}_{A}^{(t+n)}$. This formulation enables us to understand why we recognize an individual as an individual through time, even if the biochemical constituents of a biological individual are constantly replaced (cf. the "Ship of Theseus" paradox).

The same formulation can be applied to the evolutionary change of genotypes and/or phenotypes within a lienage by just modifying $\mathcal{G}_{A}^{t}$ to $\mathcal{G}_{i}$, the entire relationship of the genotype and/or phenotype of a generation $i$ within a lineage of interest $\mathcal{G}$. When we consider a series of generations,

$$
\mathcal{G}: \mathcal{G}_{i}, \mathcal{G}_{i+1}, \ldots, \mathcal{G}_{i+n}
$$

then we have its meta-level free category $\mathcal{C}(\Gamma)$, where succeccive pairs of generations share at least some part of their genotypes and/or phenotypes, respectively, but distant pairs of generations do not necessarily have shared parts.

This formulation contrives to descrive dynamic gain and loss of characters, retaining identity through generation as a lineage. At the same time, it captures autonomous aspect of life because the graph $\mathcal{G}$ represents a certain network that includes self-mediational relationships.

## Implications and prospects

In this paper, we discussed a theoretical possibility of integrating the autonomous and evolutionary aspects of life based on category theory. As our model consists of selfmediating monoids and their meta-level free category, reflecting the automonous and evolving aspecs of life, respectively, it implies that living systems are not just simple and static monoids but monoids with rich and dynamic selfmeditation. They dynamically alternate their internal subsystems in response to their environment. Such characteristic is surely not found in non-living self-mediating systems such as candle flames and typhoons.

## Acknowledgements

This work is supported by JSPS Grants-in-Aid for Scientific Research (Grant Number: JP24H01538 for D.G.S.).

## References

Bitbol, M. and Luisi, P. L. (2004). Autopoiesis with or without cognition: defining life at its edge. Journal of the Royal Society, Interface / the Royal Society, 1(1):99-107.

Bourgine, P. and Stewart, J. (2004). Autopoiesis and cognition. Artificial life, 10(3):327-345.

Eilenberg, S. and Maclane, S. (1945). General theory of natural equivalences. Transactions of the American Mathematical Society, 58:231-294.

Hirota, R., Saigo, H., and Taguchi, S. (2023). Reformalizing the notion of autonomy as closure through category theory as an arrow-first mathematics. In ALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artificial Life Conference. MIT Press.

Kauffman, S. and Roli, A. (2021). The world is not a theorem. Entropy, 23(11).

Maturana, H. R. and Varela, F. J. (1980). Autopoiesis and Cognition: The Realization of the Living. Springer, Dordrecht.

McGregor, S. and Virgo, N. (2011). Life and its close relatives. In Advances in Artificial Life. Darwin Meets von Neumann, Lecture notes in computer science, pages 230-237. Springer Berlin Heidelberg, Berlin, Heidelberg.

Montévil, M. (2019). Possibility spaces and the notion of novelty: from music to biology. Synthese, 196(11):4555-4581.

Mossio, M. and Moreno, A. (2010). Organisational closure in biological organisms. History and philosophy of the life sciences, 32(2-3):269-288.

Plantec, E., Hamon, G., Etcheverry, M., Oudeyer, P.-Y., MoulinFrier, C., and Chan, B. W.-C. (2023). Flow-lenia: Towards open-ended evolution in cellular automata through mass conservation and parameter localization. In The 2023 Conference on Artificial Life, page 131. MIT Press.

Ruiz-Mirazo, K., Peretó, J., and Moreno, A. (2004). A universal definition of life: autonomy and open-ended evolution. Origins of life and evolution of the biosphere: the journal of the International Society for the Study of the Origin of Life, 34(3):323-346.

Suzuki, D. G. and Saigo, H. (2025). A category-theoretic interpretation of the homology concept in biology. PhilSci-Archive, https://philsci-archive.pitt.edu/25010/.

Taylor, T., Bedau, M., Channon, A., Ackley, D., Banzhaf, W., Beslon, G., Dolson, E., Froese, T., Hickinbotham, S., Ikegami, T., McMullin, B., Packard, N., Rasmussen, S., Virgo, N., Agmon, E., Clark, E., McGregor, S., Ofria, C., Ropella, G., Spector, L., Stanley, K. O., Stanton, A., Timperley, C., Vostinar, A., and Wiser, M. (2016). Open-ended evolution: Perspectives from the OEE workshop in york. Artificial life, 22(3):408-423.

Witkowski, O. and Ikegami, T. (2019). How to make swarms openended? Evolving collective intelligence through a constricted exploration of adjacent possibles. Artificial life, 25(2):178197.

# Evolving Interaction Protocols for Open-ended Collective Innovation 

Eleni Nisioti ${ }^{1}$, Akhi Mocherla ${ }^{2}$, Jérémy Perez ${ }^{2}$, Clément Moulin-Frier ${ }^{2}$, Pierre-Yves Oudeyer ${ }^{2}$, Maxime Derex ${ }^{3}$, Cédric Colas ${ }^{2,4}$<br>${ }^{1}$ IT University of Copenhagen ${ }^{2}$ Inria, Bordeaux, France<br>${ }^{3}$ CNRS, Toulouse, France ${ }^{4}$ MIT, Boston, United States enis@itu.dk


#### Abstract

Human collective innovation is a remarkably open-ended process that heavily relies on the complex social dynamics of human groups. Lab and field studies with human groups have revealed that increasing group connectivity speeds up innovation in the short-term but reduces diversity within the collective, negatively impacting continual innovation. Thus, intermediate level of communication help groups achieve a better balance between exploration and exploitation. Computational studies have confirmed this in groups of evolving agents, reinforcement learning agents, and Large Language Models (LLMs), highlighting the key role of collective dynamics in engineered multi-agent systems. Despite this, systematic approaches for optimising how groups interact remain underdeveloped. In this work, we propose an approach for designing interaction protocols (IPs) that govern who communicates with whom, what is communicated, and when. We empirically demonstrate that groups solving a creative game under our evolved IPs outperform groups employing a simpler dynamic connectivity.


## Submission type: Late breaking abstract

## Introduction

In exploratory domains such as science, art, and design, progress emerges not from achieving predefined objectives but from accumulating novel and meaningful discoveries (13). Lab and field studies of human collective innovation have shown that a group's exploration and, thus, innovation abilities critically depend on how individuals communicate with each other ( $4 ; 8 ; 7$ ). For example, increasing group connectivity speeds up innovation in the short-term but reduces diversity within the collective, negatively impacting longterm innovation. Partially connected groups thus accumulate the most innovations in deceptive search spaces. Computational studies have confirmed this in groups of evolving agents (6; Cantor et al.), reinforcement learning agents (9), and Large Language Models (LLMs) (10), highlighting the key role of collective dynamics in engineered multi-agent systems. Despite this, systematic approaches for optimising how groups interact remain underdeveloped. In this work, we propose an approach for designing interaction protocols
(IPs) that govern who communicates with whom, what is communicated, and when. We empirically demonstrate that groups exploring innovation landscapes under our evolved IPs outperform groups employing a simpler dynamic connectivity.

Similarly to past computational studies $(9 ; 10)$, we use the text-based game Little Alchemy 2 (LA2) ${ }^{1}$ as a test-bed of collective innovation. LA2 defines a large, hand-designed database of crafting rules. For example, combining 'fish' and 'human' results in 'mermaid'. Agents start with a basic inventory and an instruction to try combinations leading to discovering many items. Beyond this, the game has no scoring or rewards, making LA2 an open-ended, creative game.

## Method

We employ groups of agents, each playing independently and accumulating words in its inventory over a fixed horizon. Agents collect memories of past successful and unsuccessful combinations that can guide exploration and be shared with others according to the current IP. We refer to the sharing agent as the teacher and the receiving as the learner. When a memory of a valid combination is shared, the resulting item becomes available to the learner. We focus on optimizing collective performance, measured by the total unique items in the group. IPs are implemented as Python classes with a share_memories(i_step, agent_states) method returning a dictionary of memory transmissions. In the following step, agents observe transmitted memories and may use any involved items.

To explore the IP space systematically, we employ a Quality-Diversity (QD) algorithm (11) discovering repertoires with high performance and behavioral diversity. We maintain an archive of IPs, each evaluated via multiple trials. Similarly to previous works (5), our approach follows the Novelty Search with Local Competition (NS-LC) paradigm and employs LLMs within QD for solution generation and novelty estimation. Candidates are generated by prompting the LLM in three modes: a) conditioned on simple example

[^6]![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-121.jpg?height=540&width=1572&top_left_y=278&top_left_x=242)
Figure 1: (Left) Overview of the framework used to evolve Interaction Protocols for groups of agents playing the Little Alchemy 2 game. The system iteratively generates new IPs using a language model (LLM), evaluates their performance, and maintains an archive of candidate solutions. Candidate IPs are debugged and tested, then evaluated for fitness and novelty relative to archived solutions. The archive is updated based on fitness, and novel or improved protocols are used to guide further LLM generations, enabling continual improvement and diversity in discovered solutions. (Right) Comparison of the performance of an evolved IP to dynamic and fully-connected IPs from past studies.

IPs we designed, b) improving an existing IP, and c) differing from a set of existing IPs. Novelty is estimated by embedding protocols with OpenAI's text-embedding-ada-002 and finding three nearest neighbors. An LLM reviews the candidate alongside neighbors and makes binary inclusion decisions. Accepted protocols join the archive; rejected ones replace their nearest neighbor only if fitter. This ensures diversity and competitiveness without hand-crafted behavioral descriptors. Figure 1 illustrates our pipeline for evolving IPs.

We have implemented different agent types, such as a) stochastic agents picking random inventory items, b) empowerment agents choosing combinations yielding the most empowering items (those participating in most combinations), which have been shown to fit human player behavior in LA2 (2), and c) LLM agents using an LLM to select actions based on a short game description plus current inventory and memories at each step.

## Results

To discover a well-performing IP, we ran our evolutionary algorithm for 50 generations on a group of 10 empowerment agents playing LA2 for a horizon of 500 steps. We compare its performance to two baseline IPs: a) a fully-connected IP where all agents share memories at every timestep, and b) a dynamic IP where agents start in sub-groups of two and immigrate with probability 0.01 over 5 steps (we tuned these parameters via grid-search). This dynamic IP has been shown to outperform the fully connected one in past human lab (4) and computational studies (9; 10). Figure 1 shows how the group inventory size evolves over 500 steps averaged across 10 trials, where we see that our evolved IP sig-
nificantly outperforms both baselines. The fully-connected IP performs well initially but is quickly out-competed by the others. The evolved IP exhibits much lower variance than the dynamic IP, reflecting its ability to leverage game context. In this specific run for example, the analyzed IP evolved to estimate item rarity and prioritize exchanging memories offering novel items to learners.

## Implications and future work

The creativity of open-ended, evolutionary processes has long served as an inspiration and source of scientific progress for researchers in artificial life (12; 1). Here we propose a framework for guiding a complex collective towards higher levels of creativity by appropriately shaping how the autonomous agents interact. Our framework enables scalable exploration of interaction protocol spaces in symbolic open-ended domains. By integrating programlevel search, pretrained embeddings, and LLM-mediated novelty assessment, we remove the need for hand-designed behavioral descriptors or extensive human evaluation. Our preliminary study shows that evolved IPs can out-perform well-known IPs from prior work.

Next, we will analyze and improve the framework to ensure the archive contains diverse solutions and infer highlevel properties of effective IPs. We also plan to study groups of diverse agent types, including hybrid humanartificial collectives. This work lays the foundations for principled design of collective intelligence systems in exploratory domains where traditional optimization falls short.

## References

(2020). The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities. Artificial Life, 26(2):274-306.

Brändle, F., Stocks, L. J., Tenenbaum, J. B., Gershman, S. J., and Schulz, E. (2023). Empowerment contributes to exploration behaviour in a creative video game. Nature Human Behaviour, 7(9):1481-1489.

Cantor, M., Chimento, M., Smeele, S. Q., He, P., Papageorgiou, D., Aplin, L. M., and Farine, D. R. Social network architecture and the tempo of cumulative cultural evolution. page 9.

Derex, M. and Boyd, R. (2016). Partial connectivity increases cultural accumulation within groups. Proceedings of the National Academy of Sciences, 113(11):2982-2987.

Faldor, M., Zhang, J., Cully, A., and Clune, J. (2025). Omni-epic: Open-endedness via models of human notions of interestingness with environments programmed in code.

Lazer, D. and Friedman, A. (2007). The Network Structure of Exploration and Exploitation. Administrative Science Quarterly, 52(4):667-694.

Mason, W. A., Jones, A., and Goldstone, R. L. (2008). Propagation of innovations in networked groups. Journal of Experimental Psychology: General, 137(3):422-433.

Migliano, A. B., Battiston, F., Viguier, S., Page, A. E., Dyble, M., Schlaepfer, R., Smith, D., Astete, L., Ngales, M., GomezGardenes, J., Latora, V., and Vinicius, L. (2020). Huntergatherer multilevel sociality accelerates cumulative cultural evolution. Science Advances, 6(9):eaax5913.

Nisioti, E., Mahaut, M., Oudeyer, P.-Y., Momennejad, I., and Moulin-Frier, C. (2022). Social Network Structure Shapes Innovation: Experience-sharing in RL with SAPIENS. arXiv:2206.05060 [cs].

Nisioti, E., Risi, S., Momennejad, I., Oudeyer, P.-Y., and MoulinFrier, C. (2024). Collective Innovation in Groups of Large Language Models. MIT Press.

Pugh, J. K., Soros, L. B., and Stanley, K. O. (2016). Quality Diversity: A New Frontier for Evolutionary Computation. Frontiers in Robotics and AI, 3.

Soros, L. B., Adams, A. M., Kalonaris, S., Witkowski, O., and Guckelsberger, C. (2024). On Creativity and OpenEndedness. MIT Press.

Soros, Joel Lehman, L. K. O. S. (2017). Open-endedness: The last grand challenge you've never heard of.

# Attack of the Clones: Importance of Diversity in Genetic Programming Hyper-Heuristic 

Marko Đurasević ${ }^{1}$<br>${ }^{1}$ University of Zagreb Faculty of Electrical Engineering and Computing marko.durasevic@fer.hr


#### Abstract

Hyper-heuristic $(\mathrm{HH})$ methods are becoming increasingly popular as they can generate new and appropriate heuristics for various problem variants. However, the method suffers from the gradual loss of diversity during the optimisation process, which can potentially have a negative effect on the performance of the algorithm. This study investigates the change of diversity during evolution and applies simple methods used to mitigate the problem of duplicate solutions appearing. The obtained results demonstrate that removal of duplicate solutions does not improve the performance of the algorithm. This can be interpreted as either that GP is resilient to the occurrence of duplicates or that the considered methods for improving diversity are not sophisticated enough to direct the search into promising directions.


## Submission type: Late Breaking Abstracts

## Introduction

Evolutionary computing (EC) has become an important paradigm for solving various optimisation problems (Đurasević and Jakobović, 2022). Within that field, an interesting concept emerged, denoted as a hyper-heuristics (HHs) (Burke et al., 2013). Unlike in traditional EC methods, where a single problem is solved, a HH is not used to solve a single problem, but rather to generate a novel heuristic suitable for solving a wide range of problems. In that way the problem is not being solved directly, but rather indirectly by trying to find a method appropriate for solving the problem. The most important representative of HHs is genetic programming (GP), which has been applied for generating heuristics for various problems (Branke et al., 2016).

Even though much research dealing with HHs has been performed, most of it was focused on improving the performance of the method, and only a scarce number of studies focused on investigating the evolutionary process of evolving new heuristics (Nguyen et al., 2017). However, understanding the evolutionary process of HHs is critical to be able to identify the weaknesses of the methods and propose ways of mitigating them.

One important problem that often occurs in GP and similar methods is that the diversity of the population decreases during the evolution (Poli et al., 2008). Working on a diverse set of potential solutions is of critical importance for EC methods, as it allows them to more thoroughly traverse the search space and avoid being trapped in local optima. Therefore, a lower diversity in the population can negatively affect the evolutionary process and the performance of GP. The most common way in which the diversity in the population decreases is by the appearance of duplicate solutions. The appearance of duplicate solutions inevitably leads to the situation that similar individuals are produced by genetic operators, and as such the search is confided to a small area of the solution space. However, the influence of duplicate solutions in HHs and the effect of removing duplicates was until now not investigated. This paper performs a brief analysis of the influence of duplicates on HH performance and the effect of simple techniques for mitigating the problem.

## Genetic Programming as a Hpyer-Heuristic

GP is an EC method that evolves a set (population) of random solutions, modifying them by iteratively applying various operators (crossover, mutation) until a certain termination condition (Koza, 2010). As such, GP is similar to genetic algorithms, with the important distinction that GP uses a solution representation as an expression tree. This expression tree consists out of function and terminal nodes, where function node represent different mathematical operators, and terminals represent problem specific information.

The expression that GP evolves can be used as a heuristic to solve various combinatorial optimisation problems. This heuristic incrementally constructs the solution, by selecting the most appropriate decision at each decision moment. This is performed by applying the expression on each valid decision at the current moment, and selecting the decision that received the highest (or lowest) value. Thus, the expression that GP evolves is used to rank all the decisions at the current moment and select the best one. To be able to rank the individual decisions, GP must be provided with relevant information about the problem in the form of terminal nodes.

## Methodology

To test the influence of duplicate removal in the context of HHs , three variants of GP will be investigated. The first, denoted simply as GP, represents the standard GP method without any duplicate removal. It represents the standard GP variant in which the method iteratively selects three individuals from the population randomly, performs the crossover on them to produce the child individual which is additionally mutated, and then replaces the worst of the selected individuals with the new one. The second method uses an additional penalty term added to the fitness function. This term penalises duplicate individuals depending on the number of duplicates of this individuals in the population. This method will be denoted as GP-P-X, where X represents the penalty factor used to penalise duplicate individuals. Factors of 1000 and 5000 were used to investigate its influence. This method does not ensure that duplicates are really removed, but rather tries to leverage the evolutionary process to do reduce the number of duplicates. Therefore, a third method is also used, which does not allow a duplicate solution to be added to the population, but rather repeats the genetic operators until a unique solution is created. In this case, the equality of the solutions can be checked in two ways. The first one is to compare whether two expressions are syntactically equal. This variant will be denoted as GP-EXP. Although the method is efficient, it is possible that two expressions generate the same solutions. Thus, a second variant, GP-SOL, is used, which determines that two individuals are equal if they achieve the same fitness.

## Experimental study

In order to investigate the influence of duplicate solutions on HHs, a simple problem called the container relocation problem (CRP) was considered (Tanaka and Takii, 2016). In this problem, containers have to be retrieved in a given order. However, due to limited space, they are arranged into stacks, and only the topmost container can be retrieved from the yard. If the container that needs to be retrieved next, called the target container, is not on top of its stack, it must be relocated to another stack to free up the target container. More details on how GP is used to generate heuristics for this problem are provided in (Đurasević and Đumić, 2022).

Figure 1 outlines the performance of the tested methods. As can be seen from the results, there is very little difference between the examined .methods. The Kruskall-Wallis test confirmed that these differences are not significantly different, thus we can conclude that all three of them performed equally well. As such, the removal of duplicates did not have any influence on the solution quality. Although surprising, this likely means that due to the nature of crossover and mutation in GP, which are quite destructive, even in presence of duplicate solutions, the algorithm can obtain good solutions.

To gain a notion on how the number of duplicate solutions changes over time, Figure 2 denotes the percentage

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-125.jpg?height=300&width=835&top_left_y=269&top_left_x=1078)
Figure 1:

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-125.jpg?height=304&width=826&top_left_y=699&top_left_x=1082)
Figure 2: Change of unique solutions during evolution.

of unique individuals in the population. The figure outlines only GP and GP-P, as the other two methods would have no duplicates. We see that at the start of the evolutionary process there the number of unique solutions starts rising, probably until a point at which a quite good solution is obtained. After that, the number of unique solutions gradually decreases. However, we see that the decrease is less pronounced when the penalty term is used, meaning that it does have an effect on the population structure.

## Conclusion

The obtained results demonstrate that simple methods for removal of duplicate solutions lead to no improvement in solution quality. Such a result can be interpreted in two ways. The first would be that the evolutionary process in GP is powerful enough to produce diverse solutions even in cases with many duplicates. Another interpretation would be that the examined methods considered only exact similarity, which might be too strict to achieve any improvement. In an extended study the goal would be to define a similarity measure between the solutions which would be used to eliminate not only duplicate solutions, but also solutions which are similar by their behaviour or structure.

## Acknowledgements

This work was supported by the Croatian Science Foundation under the project number IP-2022-10-5398, the European Union - NextGenerationEU under the grant NPOO.C3.2.R2-I1.06.0110., the Spanish Government under projects MCINN-23-PID2022 and TED2021-131938BI00, and by the Principality of Asturias under grant IDE/2024/000696.

## References

Branke, J., Nguyen, S., Pickardt, C. W., and Zhang, M. (2016). Automated design of production scheduling heuristics: A review. IEEE Transactions on Evolutionary Computation, 20(1):110-124.

Burke, E. K., Gendreau, M., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., and Qu, R. (2013). Hyper-heuristics: a survey of the state of the art. Journal of the Operational Research Society, 64(12):1695-1724.

Koza, J. R. (2010). Human-competitive results produced by genetic programming. Genetic Programming and Evolvable Machines, 11(3):251-284.

Nguyen, S., Mei, Y., and Zhang, M. (2017). Genetic programming for production scheduling: a survey with a unified framework. Complex \& Intelligent Systems, 3(1):41-66.

Poli, R., Langdon, W. B., and McPhee, N. F. (2008). A field guide to genetic programming.

Tanaka, S. and Takii, K. (2016). A faster branch-and-bound algorithm for the block relocation problem. IEEE Transactions on Automation Science and Engineering, 13(1):181-190.

Đurasević, M. and Jakobović, D. (2022). Heuristic and metaheuristic methods for the parallel unrelated machines scheduling problem: a survey. Artificial Intelligence Review, 56(4):3181-3289.

Đurasević, M. and Đumić, M. (2022). Automated design of heuristics for the container relocation problem using genetic programming. Applied Soft Computing.

# Why do Trees Grow? An Investigation of Bloat in Genetic Programming Hyper-Heuristics 

Marko Đurasević ${ }^{1}$<br>${ }^{1}$ University of Zagreb Faculty of Electrical Engineering and Computing marko.durasevic@fer.hr


#### Abstract

Automated generation of heuristics with genetic programming has attracted a lot of attention in recent years. This approach allows that new and efficient heuristics are generated for various combinatorial optimisation problems. However, GP suffers from a serious problem called bloat, due to which the solutions grow significantly in size without a difference in quality. Although many techniques have been proposed to mitigate the effect of bloat, we are still far from understanding its causes. This study investigates how the solution size influences the potential of generating better individuals during genetic operators, trying to find the cause for bloat in a larger effectiveness of genetic operators.


## Submission type: Late Breaking Abstracts

## Introduction

Genetic programming (GP) (Poli et al., 2008) is an evolutionary computing method commonly applied in the literature as a hyper-heuristic (HH) (Branke et al., 2016). This means that unlike other traditional optimisation methods that solve a concrete problem, GP is not oriented towards solving a single problem, but rather on obtaining a heuristic that can be utilised to solve a wide range of problems efficiently. Over the years, GP has found application as a HH in various problem domains, demonstrating its popularity (Nguyen et al., 2017; Zhang et al., 2023).

However, several problems of GP became evident over the years. One common problem is the occurrence of bloat, the growth in solution size without any evident improvement in performance. A long open question is why this phenomena occurs and what an effect it has on the evolutionary process. Even though bloat does not necessarily lead to a lower performance of GP, larger solutions are usually not preferred since they are more difficult to interpret and are less computationally efficient. As a result, many methods have been designed to mitigate the problem of bloat (Poli et al., 2008).

In this study, however, we are not interested in resolving the issue, but rather understanding what kind of effect it has on the evolutionary process when GP is applied as a HH . More specifically, we investigate how the size of the
solutions affects the potential of genetic operators to obtain solutions that are equally good or better than the parent solutions. In this way, we examine whether bloat has a direct effect on the survival of better individuals and whether this could be the potential reason that larger individuals are preferred.

## Genetic Programming

GP works similarly to genetic algorithms, starting with a random population of solutions, which are iteratively modified through the applications of genetic operators such as crossover and mutation. The main difference between these two comes from the fact that GP represents solutions as expression trees of variable size. Usually, the size of the solution is only restricted by a maximum depth, allowing GP to find the solution of an appropriate size for solving the problem. The expression tree consists of two types of nodes. Function nodes represent the inner nodes of the expression tree, acting as a glue to combine smaller expressions into larger ones. Terminal nodes represent problem specific information that is combined in order to solve the considered problem.

Although GP was traditionally applied dominantly for symbolic regression problems, in recent years it became the leading method used as a HH . This means that instead of finding a solution for a single concrete problem instance, GP generates a heuristic that can be used to solve an number of problems. This heuristic designed by GP works in a way that it constructively builds the solution by determining the decision that needs to be performed at each decision moment. The expression generated by GP is used to determine the decision that should be selected. Therefore, GP needs to be provided with terminal nodes that describe the problem well and that can be used by GP to construct an effective expression.

## Experimental Study

In order to examine the influence of bloat on the success of generating better individuals, the container relocation problem (CRP) is considered Caserta et al. (2012). In this prob-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-129.jpg?height=437&width=663&top_left_y=274&top_left_x=242)
Figure 1: Change of solution size (number of nodes) during evolution.

lem, containers are arranged into stacks, and have to be retrieved by a given order. However, a container can only be retrieved if it is located at the top of its stack. If not, containers above it must be relocated to other stacks. A standard GP method with three tournament selection is applied to generate heuristics for the considered problem. Details about applying GP to generate heuristics for this problem are provided in Đurasević and Đumić (2022). In this study, the analysis will be conducted for maximum tree depths between 3 and 13 .

First, Figure 1 demonstrates the effect of bloat on GP solution size for various depths. It is clear from the figure that the maximum tree depth used directly affects the size of the generated expressions, and that these expressions grow until a certain level. This clearly demonstrates that GP is prone to evolving larger and larger expressions.

Figure 2 shows the number of child individuals obtained after crossover and mutation that are better than bot of their parents. This number is plotted against the number of function evaluations. The figure demonstrates that regardless of the maximum tree depth the patterns that are observed are similar. At first, the number of better individuals obtained is decreasing, which is likely due to the population consisting of random solutions. But very soon, when a certain pool of good solutions are obtained, this number increases and then slowly decreases as the evolution process continues. This means that in later generations it is more difficult to obtain better solutions than the parent solutions. This is expected, as there is less possibility to obtain better solutions. Furthermore, we also see that as the expression depth increases, that the number of better children is higher. This means that the larger the expressions are, the easier it is to obtain solutions batter than their parents. This is clear since the solution space is larger, and there are more combinations to represent good solutions.

Figure 3 represents another view on the problem. In this case, we plot the number of solutions that obtained the same performance as the better of the two parents. This means that the obtained solution solves the problem in the same way as

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-129.jpg?height=245&width=666&top_left_y=274&top_left_x=1160)
Figure 2: Number of children with a better fitness compared to the best parent.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-129.jpg?height=243&width=666&top_left_y=696&top_left_x=1160)
Figure 3: Number of children with an equal quality as their parents.

the better of the two parents. The pattern of this figure can be seen to be the opposite of the previous one. This means, as it becomes less likely to obtain better child solutions compared to their parents, it becomes more likely to obtain solutions that have the same behaviour as the better parent. Again, this number is larger as the tree depth is larger. The reason for this is simply that smaller trees are more sensitive to the change in their structure, whereas in larger expressions this change might not be

## Conclusion

The analysis performed in this study demonstrated that the increase in the size of the individuals increases the likelihood of GP generating equally good or better individuals compared to their parents. Therefore, bloat can be regarded as a defensive mechanism, which enables that the genetic material of fitter individuals can more easily be transferred to the next generation.

The preliminary analysis performed in this study opens several follow up research directions. Most importantly, the current analysis was aimed only on analysing the size of te expressions, but other things, like the number of nodes that do not affect the output of the expression, should be considered. Furthermore, it would also be interesting to see how the different mechanisms to prevent bloat influence the evolutionary process.

## Acknowledgement

This work was supported by the Croatian Science Foundation under the project number IP-2022-10-5398, the European Union - NextGenerationEU under the grant NPOO.C3.2.R2-I1.06.0110.

## References

Branke, J., Nguyen, S., Pickardt, C. W., and Zhang, M. (2016). Automated design of production scheduling heuristics: A review. IEEE Transactions on Evolutionary Computation, 20(1):110-124.

Caserta, M., Schwarze, S., and Voß, S. (2012). A mathematical formulation and complexity considerations for the blocks relocation problem. European Journal of Operational Research, 219(1):96-104.

Nguyen, S., Mei, Y., and Zhang, M. (2017). Genetic programming for production scheduling: a survey with a unified framework. Complex \& Intelligent Systems, 3(1):41-66.

Poli, R., Langdon, W. B., and McPhee, N. F. (2008). A field guide to genetic programming.

Zhang, F., Mei, Y., Nguyen, S., and Zhang, M. (2023). Survey on genetic programming and machine learning techniques for heuristic design in job shop scheduling. IEEE Transactions on Evolutionary Computation, page 1-1.

Đurasević, M. and Đumić, M. (2022). Automated design of heuristics for the container relocation problem using genetic programming. Applied Soft Computing.

# Swarm2Algo: Towards Distilling Emergent Collective Behaviors as Interpretable Distributed Algorithms 

Devendra Parkar ${ }^{1}$, Raylene A. Faerber ${ }^{1}$, Hillary Li ${ }^{1}$, Ping-Yu Duan ${ }^{1}$, and Joshua J. Daymude ${ }^{1}$<br>${ }^{1}$ Biodesign Center for Biocomputing, Security and Society and School of Computing and Augmented Intelligence Arizona State University, Tempe, AZ, United States dparkar1 @ asu.edu


#### Abstract

Surprising complex emergent behavior can arise from simple, local interactions among agents. Parkar et al. (ALIFE 2024) showed that stochastic algorithms that govern local interactions of such agents can be discovered for desired collective emergent behaviors, and put forth a framework called EvoSOPS. However, the algorithms discovered by EvoSOPS are not human-interpretable. Thus, just as the field of AI and ML grapples with the questions of interpretability, the field of complex systems and collective behaviors must tackle this important issue-especially for general methods which apply broadly to many behaviors like EvoSOPS. As such, we propose to study this problem using a multi-pronged approach, in which we use a variety of evolutionary search methods to find diverse solution algorithms. Analyzing these algorithms, we look to derive inner structures and draw out course-grained rule-based interpretable algorithms. We call this new approach Swarm2Algo framework and demonstrate that diverse evolutionary search methods indeed result in diverse solution algorithms.


## Submission type: Late Breaking Abstracts

## Introduction

The beauty of starling murmurations (King and Sumpter, 2012; Goodenough et al., 2017) and the efficiency of ant colonies (Flanagan et al., 2012; Cornejo et al., 2014) are examples of sophisticated collective behaviors that emerge from local interactions. Beyond biology, such behaviors can be used to design distributed systems such as swarm robots, autonomous vehicular networks, or the Internet.

The initial study into stochastic algorithms for collective behaviors was based in self-organizing particle systems (SOPS), where each particle in the system is simple-having no persistent memory, interacting locally, and moving over a discrete lattice (Derakhshandeh et al., 2014; Daymude et al., 2019). The individual particles all perform the same distributed algorithm. Using methods from stochastic theory, several SOPS algorithms have been found for different collective behaviors (Cannon et al., 2016; Andrés Arroyo et al., 2018; Savoie et al., 2018; Cannon et al., 2019; Li et al., 2021; Kedia et al., 2022; Oh et al., 2023). However, the
stochastic approach used to develop these algorithms requires extensive bespoke analysis for each new behavior considered (Cannon et al., 2016). Moreover, the stochastic approach only yields one algorithm per task-offering scarce insight into the diversity of the local interactions.

Inspired by this paradigm, we introduced EvoSOPS framework which uses a genetic algorithm (GA) to search a discrete genomic representation space to find stochastic, distributed SOPS algorithms that achieve mathematically specified collective behaviors. The SOPS algorithms map neighborhood configurations of a particle to probabilities of movement for that particle and the genomic representation uses integer-valued alleles $i \in\{0, \ldots, 10\}$ corresponding to movement probabilities $2^{-i}$. To evaluate the algorithms, fitness functions rewarded the achieved collective behavior's scalability across varying particle numbers and robustness from randomized initial conditions. EvoSOPS found algorithms for four collective behaviors: aggregation, where particles gather compactly; phototaxing, where particles simultaneously aggregate and move towards a light source; separation, where heterogeneous particles aggregate as a whole and by color class; and object coating, where particles form as many even layers as possible around a connected but arbitrarily shaped object. An analysis of the best-fitness genomes from different runs for aggregation also illustrated how to discover interpretable algorithms.

However, the GA used in EvoSOPS is restricted to searching for a limit set of solution algorithm due to the discrete set of probabilities used to represent the stochastic algorithms. Although EvoSOPS finds diverse solutions it cannot explore all possible high performing regions of algorithm search space due to this limitation. Furthermore, the attempts to interpret the solution algorithms are limited to aggregation behavior which has a stochastic theory based algorithm to compare against and primarily succeeds due to this comparison. An approach to finding interpretable algorithms to behaviors without a theoretical algorithm (such as object coating) is missing. And speaking broadly, despite long standing research into extracting higher-order patterns in emergent behaviors (Crutchfield and Mitchell, 1995), dis-
tilling emergent collective behaviors as interpretable local interaction rules is not yet looked at.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-133.jpg?height=681&width=652&top_left_y=386&top_left_x=255)
Figure 1: UMAP embeddings of all generations' genomes for three independent CMA-ES and GA runs per collective behavior. CMA-ES and GA runs are shown as a distinct color map, gray and pink respectively, with higher color intensity indicating higher fitness.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-133.jpg?height=332&width=653&top_left_y=1409&top_left_x=248)
Figure 2: An algorithm's genome representation is a list of real-valued genes indexed by groups of extended neighborhoods. For example, the aggregation genome groups extended neighborhoods by the number of neighbors in their back (blue), middle (green), and front (yellow) regions. The alleles correspond to movement probabilities.

## Swarm2Algo framework

We introduce Swarm2Algo framework which aims to bridge this gap and focuses on finding interpretable rule-based algorithms for emergent collective behaviors, especially for behaviors which do not yet have interpretable theoretically established algorithms. We list below the steps of Swarm2Algo framework,

1. Define a fitness measure (objective function) for a desired collective behavior
2. Create a genomic representation based on the loci of a single particle and states it can have (refer to Figure 2 for an example).
3. Utilize a suite of evolutionary search methods to search for solution algorithms
4. Juxtapose and compare solution algorithms to extract interpretable rule-based algorithm(s) for the desire collective behavior

The key assumptions of the framework are:

1. Diverse evolutionary methods help in finding diverse high fitness solution regions in algorithm space.
2. Comparing and juxtaposing diverse solutions aids in understanding the latent structure within the genomic structure representation of the algorithms (ie. gene interactions within the genome). Which in turn helps in finding most important genes of the genomic structure (patterns which are preserved and are most influential to the emergent desire behavior) to extract a low-dimensional and interpretable algorithm. This assumption was validated for Aggregation behavior in (Parkar et al., 2024) where they compared diverse best fitness genomes across different runs and stochastic theory algorithm.

Table 1: Fitness values for the best-fitness genomes for CMA-ES and GA methods.
| Method | Aggregation | Phototaxing | Separation | Object-Coating |
| :--- | :---: | :---: | :---: | :---: |
| CMA-ES | 0.99 | 0.99 | 0.84 | 0.84 |
| GA | 0.98 | 0.99 | 0.97 | 0.83 |


CMA-ES and Results. As a concrete first step, we verify through our experiments the first key assumption of the framework. We utilize CMA-ES (covariance matrix adaptation based evolutionary search) method due to its ability to work efficiently with real-valued genomic representations and plug the gap left by the EvoSOPS GA's limited expressivity. The CMA-ES's real-valued genomic representation is highlighted in the Figure 2. Details about the CMA-ES method implementation can be found in (Hansen, 2006).

We demonstrate the efficacy of the CMA-ES method in Table 1. Next, we highlight the key result of this abstract, the CMA-ES does find diverse solution algorithms which are significantly different than algorithms found by GA as seen in Figure 1. Where regions explored by CMA-ES are markedly different than those explored by GA and this distinction becomes apparent as the complexity of the behaviors increases from left to right and top to bottom.

## Acknowledgements

This work was supported by the NSF (CCF-2312537).

## References

Andrés Arroyo, M., Cannon, S., Daymude, J. J., Randall, D., and Richa, A. W. (2018). A Stochastic Approach to Shortcut Bridging in Programmable Matter. Natural Computing, 17(4):723-741.

Cannon, S., Daymude, J. J., Gökmen, C., Randall, D., and Richa, A. W. (2019). A Local Stochastic Algorithm for Separation in Heterogeneous Self-Organizing Particle Systems. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2019), volume 145 of Leibniz International Proceedings in Informatics (LIPIcs), pages 54:1-54:22, Dagstuhl, Germany. Schloss Dagstuhl - Leibniz-Zentrum für Informatik.

Cannon, S., Daymude, J. J., Randall, D., and Richa, A. W. (2016). A Markov Chain Algorithm for Compression in SelfOrganizing Particle Systems. In Proceedings of the 2016 ACM Symposium on Principles of Distributed Computing, pages 279-288, New York, NY, USA. ACM.

Cornejo, A., Dornhaus, A., Lynch, N., and Nagpal, R. (2014). Task Allocation in Ant Colonies. In Distributed Computing, volume 8784 of Lecture Notes in Computer Science, pages 4660, Austin, TX, USA. Springer.

Crutchfield, J. P. and Mitchell, M. (1995). The evolution of emergent computation. 92(23):10742-10746.

Daymude, J. J., Hinnenthal, K., Richa, A. W., and Scheideler, C. (2019). Computing by Programmable Particles. In Flocchini, P., Prencipe, G., and Santoro, N., editors, Distributed Computing by Mobile Entities, volume 11340 of Lecture Notes in Computer Science, pages 615-681. Springer, Cham.

Derakhshandeh, Z., Dolev, S., Gmyr, R., Richa, A. W., Scheideler, C., and Strothmann, T. (2014). Amoebot - a New Model for Programmable Matter. In Proceedings of the 26th ACM Symposium on Parallelism in Algorithms and Architectures, pages 220-222, New York, NY, USA. ACM.

Flanagan, T. P., Letendre, K., Burnside, W. R., Fricke, G. M., and Moses, M. E. (2012). Quantifying the Effect of Colony Size and Food Distribution on Harvester Ant Foraging. PLoS ONE, 7(7):e39427.

Goodenough, A. E., Little, N., Carpenter, W. S., and Hart, A. G. (2017). Birds of a Feather Flock Together: Insights into Starling Murmuration Behaviour Revealed Using Citizen Science. PLoS ONE, 12(6):e0179277.

Hansen, N. (2006). The CMA Evolution Strategy: A Comparing Review. In Lozano, J. A., Larrañaga, P., Inza, I., and Bengoetxea, E., editors, Towards a New Evolutionary Computation: Advances in the Estimation of Distribution Algorithms, pages 75-102. Springer.

Kedia, H., Oh, S., and Randall, D. (2022). Local Stochastic Algorithms for Alignment in Self-Organizing Particle Systems. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2022), volume 245 of Leibniz International Proceedings in Informatics (LIPIcs), pages 14:1-14:20, Dagstuhl, Germany. Schloss Dagstuhl - Leibniz-Zentrum für Informatik.

King, A. J. and Sumpter, D. J. (2012). Murmurations. Current Biology, 22(4):R112-R114.

Li, S., Dutta, B., Cannon, S., Daymude, J. J., Avinery, R., Aydin, E., Richa, A. W., Goldman, D. I., and Randall, D. (2021). Programming Active Cohesive Granular Matter with Mechanically Induced Phase Changes. Science Advances, 7(17):eabe8494.

Oh, S., Randall, D., and Richa, A. W. (2023). Adaptive Collective Responses to Local Stimuli in Anonymous Dynamic Networks. In 2nd Symposium on Algorithmic Foundations of Dynamic Networks (SAND 2023), volume 257 of Leibniz International Proceedings in Informatics (LIPIcs), pages 6:16:23, Pisa, Italy. Schloss Dagstuhl - Leibniz-Zentrum für Informatik.

Parkar, D., Leyba, K. G., Faerber, R. A., and Daymude, J. J. (2024). Evolving Collective Behavior in Self-Organizing Particle Systems. In ALIFE 2024: The 2024 Artificial Life Conference, page 36, Copenhagen, Denmark. MIT Press.

Savoie, W., Cannon, S., Daymude, J. J., Warkentin, R., Li, S., Richa, A. W., Randall, D., and Goldman, D. I. (2018). Phototactic Supersmarticles. Artificial Life and Robotics, 23(4):459-468.

# Pheromone-mediated movement establishes spatial segregation in simulated ants 

Maisie Sadler and Seth Bullock<br>University of Bristol, UK, seth.bullock @bristol.ac.uk

Using experiments on ants from several different species, Richardson et al. (2022) demonstrate that the preferred spatial zone of a worker ant is associated with its functional role within the colony (e.g., nurses prefer to remain within a central zone whereas foragers prefer to remain outside this central zone). An agent-based simulation of a single ant shows that a combination of two simple movement mechanisms can account for this spatial segregation given two assumptions: (i) the relevant spatial zones are already established, and (ii) ants are able to detect the boundary between them.

In order to explore how these spatial zones might become established originally and by what mechanism ants might detect the boundary between them, we replicate the original agent-based simulation and extend it to model multiple ants interacting within the nest. We demonstrate that a simple pheromone deposition mechanism is sufficient to account for the spatial segregation finding. Furthermore, we explore the influence of a central brood and the presence of a nest entrance on the nature of the spatial segregation, demonstrating that a typical zone configuration observed in the original study (a stable circle-surround configuration with a third intermediate zone) can be reliably achieved under conditions featuring a central brood and a peripheral nest entrance.

## Model

Richardson et al. (2022) model an ant nest as a periodic (i.e., toroidal) hexagonal lattice with a central zone and a surrounding peripheral zone. One simulated ant moves stochastically in discrete steps of varying length and heading. The ant's fixed role (forager or nurse) and current state (active or less active) influence its movement via a set of parameters calibrated using empirical data obtained by tracking acorn ants in a laboratory nest. To simulate a colony of $n$ ants, $n$ independent single-ant simulations are run.

Four movement schemes are considered: a Correlated Random Walk (CRW), Boundary Effect Walk (BEW), Locomotion Adjustment Walk (LAW), and Boundary Effect plus Locomotion Adjustment Walk (BELA). A CRW does not take account of an ant's location within the nest and is treated as a baseline null model against which to compare
the other models, each of which do take account of the ant's role (nurse or forager) and also make use of the ant's proximity to the boundary between the two zones to influence either the turn angle of the ant (BEW), its tendency to change state from active to less active (LAW), or both (BELA). Simulated BELA walks provided the best match to the observed behaviour in real ants.

We extend this model by removing the fixed zones, simulating multiple (initially randomly placed) ants simultaenously in order to let them influence each other, and introducing pheromone deposition as a mechanism by which this influence can occur. At each simulated time step, each ant deposits one unit of pheromone in the cell that it occupies (nurses deposit nurse pheromone, foragers deposit forager pheromone), a proportion of the pheromone at each cell diffuses into adjacent cells, and pheromone in each cell decays to some extent. For simulations that feature a central brood (i.e., eggs or young that require nursing), at every time step some nurse pheromone is injected into the most central cells within the nest. For simulations that feature a nest entrance, the nest lattice is bounded rather than periodic and at every time step some forager pheromone is injected into a group of cells mid-way along the south wall of the nest.

We replace the movement rules that were used in the original study with new rules that use an ant's recent history of pheromone sensations to estimate pheromone intensity and pheromone gradient values for its preferred pheromone. These are used to drive the BELA movement rule using the parameters from the original study. Here, pheromone intensity and gradient estimates are being used as proxies for information about the whereabouts of an emergent boundary between spatial zones. Unlike the original simulation, no spatial zones are defined or imposed on the nest and no direct information about an ant's location is available to it.

Spatial segregation is measured using Lee's $L$ (a bivariate spatial correlation coefficient; Lee, 2001) applied to the pheromone distribution within the nest. A value of 1 sig nifies perfect spatial correlation between two distributions, a value of zero indicates random mixing, and a value of -1 indicates perfect spatial anti-correlation (i.e., segregation).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-137.jpg?height=385&width=1655&top_left_y=264&top_left_x=205)
Figure 1: Representative pheromone distributions for 114-agent, $10^{4}$-step simulations: (a) CRW ( $L=-0.01$ ), (b) BELA band ( $L=-0.85$ ), (c) BELA centre-surround ( $L=-0.61$ ), (d) BELA with brood (e) BELA with brood and southern entrance.

## Results

First, we replicated the results of the original study with single-ant simulations employing the original four movement rules. Close agreement was achieved on all measures reported in the original study. Then, we explored whether the pheromone-driven behaviour could result in spatial segregation. Pheromone-driven BELA walks exhibited a number of segregated zonal configurations (Fig. 1) and achieved significantly better $L$ scores than equivalent CRW simulations (Fig. 2). Two main kinds of spatial segregation were observed in toroidal nests. In one, ants organised into two bands, both running vertically across the nest, or both running horizontally (Fig. 1b), while in the other, a circlesurround configuration similar to that reported in the original study was observed (Fig. 1c). The spatial configuration of the zones evolved over time and was not anchored in place. When a central brood was introduced, however, a consistent, stable circle-surround configuration was typically achieved (Fig. 1d). The addition of a nest "entrance" (implemented as an increased quantity of forager pheromone along one edge of a bounded nest) was capable of establishing a third "intermediate" zone (Fig. 1e) reminiscent of natural nest configurations reported by Richardson et al. (2022).

## Conclusion

Here, a simple pheromone deposition mechanism is demonstrated to be sufficient to enable simulated ants to establish and maintain spatial segregation consistent with that reported in a previous empirical study. This segregation is robust to parameter choices for pheromone diffusion and decay, and agent sensory integration (not shown). Manipulating the boundary conditions of the nest and the presence/absence of a central brood and/or peripheral nest entrance influences the configuration of the spatial zones in a plausible manner. Future work will introduce foraging trips, disease and the relocation of the brood within the nest.

## Acknowledgements

We thank Nathalie Stroeymeyt and Tom Richardson for their advice. This work was supported by UKRI EPSRC Grant No. EP/Y028392/1: AI for Collective Intelligence (AI4CI).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-137.jpg?height=1202&width=791&top_left_y=822&top_left_x=1119)
Figure 2: The impact of pheromone diffusion and decay rates on segregation for BELA simulations (green) relative to a CRW baseline (orange). Successful segregation is indicated by values approaching -1. Each data point shows the mean and standard error of 50 simulations of 114 agents for $10^{4}$ time-steps. Results are plotted here for toroidal nests with no central brood or peripheral entrance (i.e., nests with periodic boundary conditions as employed in the original study being replicated). For the upper plot, $\lambda=0.0008$. For the lower plot, $D=0.14$. Spatial segregation is highly robust with respect to choice of pheromone parameters.

## References

Lee, S.-I. (2001). Developing a bivariate spatial association measure: An integration of Pearson's $r$ and Moran's I. Journal of Geographical Systems, 3(4):369-385.

Richardson, T. O., Stroeymeyt, N., Crespi, A., and Keller, L. (2022). Two simple movement mechanisms for spatial division of labour in social insects. Nature Communications, 13(1):6985.

# Evidence for Open-Ended Evolution in a Binary Cellular Automaton 

Arend Hintze ${ }^{1,2}$ and Clifford Bohm ${ }^{2}$<br>${ }^{1}$ Department of Data Analytics, Dalarna University<br>${ }^{2}$ BEACON Center for the Study of Evolution in Action, Michigan State University.<br>ahz@du.se


#### Abstract

The Outlier cellular automaton exhibits complex dynamics suggestive of self-replication. We developed a causal lineage framework that reconstructs complete developmental histories in deterministic systems, allowing rigorous identification of self-replicators through explicit parent-offspring relationships. Using this method, we discovered spontaneous multigenerational self-replicators, novel replicator variants emerging throughout extended runs, and power-law distributions in pattern emergence-signatures of unbounded growth. These findings suggest that open-ended evolution can emerge from minimal computational substrates: binary states, deterministic rules, and spatial extension, without any explicit design for replication or evolution.


## Submission type: Late Breaking Abstracts

## Introduction

Open-ended evolution-the continuous generation of novelty and complexity-has long been a central goal in artificial life (Langton, 1986; Bedau et al., 2000). While engineered cellular automata (CAs) like Langton's loops (Langton, 1984) demonstrate controlled replication, and systems like evoloops show evolutionary dynamics (Sayama and Nehaniv, 2024), these require careful design. The requirements for open-ended evolution have been extensively debated, with various conditions proposed, including unbounded diversity (Maley, 1999), novel opportunities for adaptation (Soros and Stanley, 2014), and unbounded phenotypic complexity (Taylor et al., 2016). Recent work suggests that systems exhibiting power-law dynamics in adaptation, rather than plateauing, may be genuinely open-ended (Wiser et al., 2013). However, as noted by Hintze (2019), many simple systems can satisfy formal open-endedness criteria while producing only trivial complexity, highlighting the importance of meaningful metrics for complexity and diversity (Hintze, 2019).

The Outlier rule, a binary CA discovered through novelty search rather than explicit engineering, exhibits complex emergent behaviors without built-in replication mechanisms (Yang, 2024). We investigated whether this mini-
mal system could support the three key requirements for open-ended evolution: replication, variation, and unbounded innovation-and whether it produces the kind of meaningful complexity that distinguishes truly open-ended systems from those that merely satisfy formal criteria.

## The Causal Lineage Method

To identify and analyze self-replicators in the Outlier CA, we developed a causal lineage framework that traces how patterns give rise to other patterns through the deterministic update rules. In any CA, each cell's next state depends on its neighborhood's current state. For the Outlier rule, we can determine exactly which cell states from the Moore neighborhood at time $t$ causally contributed to each alive cell at time $t+1$.

While individual cells have clear causal dependencies, patterns in CAs branch and merge dynamically-a single cluster can split into multiple disjoint clusters or multiple clusters can merge into one. To capture these complex developmental pathways and identify true self-replication, we track causal relationships at the pattern level.

We construct a directed graph where nodes represent cluster instances-specific patterns at particular spacetime coor-dinates-and edges indicate causal relationships. For each alive cell, we identify the minimal set of neighboring cells necessary for its activation, eliminating redundant dependencies. These cell-level causal links aggregate to the cluster level: cluster $B$ descends from cluster $A$ if any cells in $B$ depend on cells from $A$.

This causal ancestry graph enables us to definitively identify self-replicators as clusters producing multiple causally independent offspring. Each offspring traces back to the parent but not to sibling offspring, forming branching phylogenetic trees. The method is fully deterministic and reconstructs the complete developmental history of every pattern in the system, providing unprecedented insight into how replication emerges and evolves.

## Key Findings from the Outlier CA

Applying our method to 20,000 ticks of the Outlier rule (initialized with a simple 5 -cell cluster on a $1024 \times 1024$ grid) revealed three critical findings supporting open-ended evolution:

Finding 1: Spontaneous Self-Replicators. We identified robust self-replicating processes emerging without any explicit design for replication. To understand the significance of this finding, consider that most known CA replicators require careful engineering-specific initial patterns designed to copy themselves through predetermined sequences of states. In contrast, starting from a simple 5 -cell seed pattern, the Outlier CA spontaneously generates processes that replicate specific patterns.

The most successful replicating process involves a pattern we designate c2, which first appears at tick 3. Crucially, c2 is not itself the replicator-rather, it initiates a complex developmental sequence that eventually produces four new c2 patterns. This is analogous to how a glider in Conway's Game of Life is not just one pattern, but the entire four-step cycle that defines its existence. However, unlike a glider's simple cyclic trajectory, the c2 replication process branches into multiple pathways, with different clusters evolving semi-independently before generating multiple copies of c2. In some cases, individual branches of the replication will result in temporally isolated patterns that later converge to recreate copies of c2.

Over 10,000 ticks, we observed 2,439 instances of c 2 , which were traceable through our causal framework to the original. Of these, 344 successfully initiated their own replication processes, creating a branching phylogenetic tree spanning 15 generations with exponential growth (factor $\approx 1.5$ ). The complete replicator consists of the entire developmental lineage-hundreds of intermediate patterns forming a distributed, multi-component process where spatially separated clusters coordinate across time to recreate the original c2 pattern. This fundamentally differs from engineered replicators that maintain cohesive structure throughout copying, representing a new mode of replication emerging from the system's dynamics.

Finding 2: Novel Replicator Variants. The system continuously generates new replicator variants. We identified 18 distinct developmental pathways for c2 replication, ranging from 572 to 881 ticks-not minor variations but fundamentally different causal trajectories. Moreover, searching after tick 5,000 revealed 205 c2-like replicators, including 18 that did not descend from the original c2. These genuinely new replicators arise from the system's dynamics, demonstrating ongoing innovation rather than mere copying of existing functionality.

Finding 3: Power-Law Dynamics. Analysis over 75,000 ticks on an unbounded grid revealed power-law distributions in both pattern emergence times (AIC $=104114.5$ vs 163790.6 for hyperbolic) and maximum cluster sizes (AIC $=$
1949.6 vs 2046.1). These distributions, characteristic of systems exhibiting long-term evolution (Wiser et al., 2013), indicate the Outlier CA does not plateau but continues generating novel patterns and increasing complexity indefinitely.

## Implications for Open-Ended Evolution

These findings demonstrate that the Outlier CA satisfies the core requirements for open-ended evolution. The system exhibits: (1) self-replication through spontaneous emergence of replicating clusters, (2) heritable variation through diverse developmental pathways and novel replicator variants, and (3) unbounded growth evidenced by power-law dynamics in pattern generation.

The distributed nature of replication challenges conventional notions of individuality in artificial life. Traditional replicators like Langton's loops maintain structural cohesion throughout replication. In contrast, Outlier replicators fragment into multiple disjoint components that somehow coordinate to reconstruct the original pattern. This suggests that coherent individuality is not a prerequisite for evolution loosely coupled, distributed processes can serve as units of selection.

The emergence of these properties from such a minimal substrate - binary states, local deterministic rules, and spatial extension - suggests that open-ended evolution may be far more accessible than previously thought. The Outlier rule was not designed for replication or evolution; it was discovered through novelty search among millions of random rules. If such a simple, unengineered system can support open-ended evolution, similar phenomena may be widespread in natural and artificial systems.

This has profound implications for understanding the emergence of complexity. Rather than requiring sophisticated machinery or careful engineering, open-ended evolution appears to be a natural consequence of simple rules interacting in spatial contexts. The Outlier CA provides a tractable model for studying these fundamental mechanisms, offering insights into how minimal computational substrates can generate unbounded complexity and diversity.

Future work should explore whether other CAs discovered through automated search exhibit similar properties, investigate the conditions that enable or constrain openended evolution, and examine how these minimal models relate to biological evolution and artificial life systems. The causal lineage method presented here (and described in full in redacted) provides a rigorous framework for such investigations, enabling quantitative analysis of evolutionary dynamics in deterministic systems.

## Acknowledgments

Large Language Models (ChatGPT, Claude, and DeepSeek) were used to refine, but not create, content in this work.

## References

Bedau, M. A., McCaskill, J. S., Packard, N. H., Rasmussen, S., Adami, C., Green, D. G., Ikegami, T., Kaneko, K., and Ray, T. S. (2000). Open problems in artificial life. Artificial life, 6(4):363-376.

Hintze, A. (2019). Open-endedness for the sake of openendedness. Artificial life, 25(2):198-206.

Langton, C. G. (1984). Self-reproduction in cellular automata. Physica D: Nonlinear Phenomena, 10(1-2):135-144.

Langton, C. G. (1986). Studying artificial life with cellular automata. Physica D: nonlinear phenomena, 22(1-3):120-149.

Maley, C. (1999). Four steps toward open-ended evolution. In Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation-Volume 2, pages 1336-1343. Morgan Kaufmann Publishers Inc.

Sayama, H. and Nehaniv, C. L. (2024). Self-reproduction and evolution in cellular automata: 25 years after evoloops. Artificial Life, pages 1-15.

Soros, L. B. and Stanley, K. O. (2014). Identifying necessary conditions for open-ended evolution through the artificial life world of chromaria. In H. Sayama, J. Rieffel, S. Risi, R. Doursat, \& H. Lipson (Eds.), Artificial life 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems, pages 793-800. Citeseer.

Taylor, T., Bedau, M., Channon, A., Ackley, D., Banzhaf, W., Beslon, G., Dolson, E., Froese, T., Hickinbotham, S., Ikegami, T., et al. (2016). Open-ended evolution: perspectives from the oee workshop in york. Artificial life, 22(3):408-423.

Wiser, M. J., Ribeck, N., and Lenski, R. E. (2013). Long-term dynamics of adaptation in asexual populations. Science, 342(6164):1364-1367.

Yang, B. (2024). Emergence of self-replicating hierarchical structures in a binary cellular automaton. Artificial Life, pages 1-10.

# Spark: Modular Spiking Neural Networks 

Mario Franco ${ }^{1 *}$ and Carlos Gershenson ${ }^{1}$<br>${ }^{1}$ School of Systems Science and Industrial Enginnering. Binghamton University, USA<br>*mfrancomndez@binghamton.edu


#### Abstract

Nowadays, neural networks act as a synonym for artificial intelligence. Present neural models, although remarkably powerful, are inefficient both in terms of data and energy. Several alternative forms of neural networks have been proposed to address some of these problems; specifically spiking neural networks are suitable for efficient hardware implementations. However, effective learning algorithms for spiking networks remain elusive. Here, we present a new framework for spiking neural networks centered around the idea of modular design from the components of the neurons themselves to the entire model. The aim of this framework is to provide a common ground and accelerate spiking neural network research for continuous and unbatched learning, akin to the one animals exhibit.


## Submission type: Late Breaking Abstracts

Code available at: Spark ${ }^{1}$

## Introduction

Current artificial intelligence systems inspired by the brain have shown remarkable capabilities in most domains of human interest. Stull, the predominant flavor of neural networks is extremely inefficient, both in terms of energy and information, compared to natural neural networks. Several alternative artificial neural networks have been proposed, with spiking neural networks being the most popular among them because of the possibility of extremely efficient hardware implementations. However, spiking neural networks remain quite hard to train to perform almost any task due to their non-differentiable dynamics, although surrogate gradient approaches have shown some promise in the past (Neftci et al., 2019). Moreover, it is unclear whether a batched approach is the best way to learn efficiently using a neural network; the animal brain cannot batch information, but is very efficient nonetheless. In this work, we introduce a new framework - Spark - for spiking neural networks. Its purpose is to develop a common ground for exploring and researching unbatched and continuous inference.

[^7]
## Spark

Spark is a new framework designed to boost spiking neural network research by creating an efficient and modular ground for designing and sharing models, similar to other frameworks such as Keras and Flax. Spark was designed around the idea of modular neurons: the literature around spiking neural networks is vast and full with interesting but sometimes scattered results. For example, it has been observed that basal and apical dendrites within the same neuron exhibit different plasticity dynamics (Ebner et al., 2019); similarly, modulation plays a major role in plasticity outcomes (Fuchsberger and Paulsen, 2022). Ideally, this should be as simple as adding two sets of synapses, with possibly different rules and targets (or configurations). However, in practice, this often leads to writing an entire new model from scratch and, in the long run, an important amount of boilerplate or unreadable code.

Implementation details in machine learning are always the link between a sound theory and a functional model. As such, complex code can get in the way of good ideas. Diffusion models (Ho et al., 2020) are a simple example of this, the theory behind them is rather simple and intuitive. Still, the implementation can be rather tricky. Thus, a common stage for research may prove beneficial to explore new and more complex ideas.

In Spark, we take a rather pragmatic approach: a neuron is any entity capable of receiving and emitting spikes. What happens in between is open to interpretation. We define a set of core elements: somas (compartments), synapses, learning rules, and some auxiliary elements like conduction delays and input/output interfaces. This set of elementary components can be used to implement a wide variety of models with very few lines of code.

Unlike most modern machine learning approaches, Spark does not follow the data-batching paradigm. Although there is no doubt that animals aggregate data, to the best of our knowledge, there is no supporting evidence that databatching, as traditionally implemented, may occur in natural brains. Moreover, algorithms inspired by neuroscience research often lead to data-aggregation instead of data-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-145.jpg?height=762&width=1730&top_left_y=276&top_left_x=169)
Figure 1: A view (screenshot) of a simple model of two competing neuron populations A and B built with Spark's graph editor. The model receives a signal at the input node, transforms it into to a spike signal and feeds it to both populations. Each population contains two pools of neurons: excitatory (Ex) and inhibitory (In). Each population tries to propagate a signal (current) to the output node while, at the same time, tries to inhibit the other population. Intuitively, this model is rather simple, however a direct implementation of this model can be quite tricky and prone to error.

batching approaches. We speculate that data structure, rather than quantity, is vital for effective and efficient learning. Animals often learn from very few examples, although these examples typically unfold in time and are experienced as a sequence rather than a collection. The implicit topological structure of the experience may reveal a simpler learnable pattern than the one hinted in thousands of isolated examples.

Recurrence is a common ingredient that is widely explored in spiking networks. However, recurrence can be tricky to implement efficiently. With Spark, we seek to simplify model design at every level. However, thanks to the shift from a data-batching to a data-aggregation paradigm, it is possible to implement a fast, distributed simulation of recurrent networks at the expense of some extra memory. Spark is built on top of the JAX and Flax libraries (Bradbury et al., 2018; Heek et al., 2024), which allows for Just-In-Time compilation of entire models, drastically improving the performance of the simulation without adding any significant complexity to the interface or the code.

Additionally, Spark includes a small editor that can be used to design extremely complicated models without the need of lines of code. As mentioned above, recurrence is often explored in spiking networks and, in contrast to pure feed-forward models, describing recurrent models with many elements is significantly more challenging and prone to error. That is why we seek to alleviate this problem by means of an intuitive editor. Figure 1 shows a preview of the
graph editor used to construct a simple model of two competing populations.

Finally, we want to highlight that Spark is not a finished product, but a framework for the exploration of a different set of ideas in machine learning. The final goal of this framework is to act only as a guide for future research.

## Conclusions

We presented a new framework for spiking neural networks. Similar to other machine learning frameworks, Spark embraces modularity by design, providing a flexible yet powerful tool for spiking networks research. It is important to note that Spark does not follow the data-batching paradigm. This shift may help us explore different avenues in machine learning and help us discover/design new algorithms. We think of Spark not as a finished product but rather as a vehicle to explore a different flavor of machine learning.

## References

Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., and Zhang, Q. (2018). JAX: composable transformations of Python+NumPy programs.

Ebner, C., Clopath, C., Jedlicka, P., and Cuntz, H. (2019). Unifying long-term plasticity rules for excitatory synapses by modeling dendrites of cortical pyramidal neurons. Cell Reports, 29(13):4295-4307.e6.

Fuchsberger, T. and Paulsen, O. (2022). Modulation of hippocampal plasticity in learning and memory. Current Opinion in Neurobiology, 75:102558.

Heek, J., Levskaya, A., Oliver, A., Ritter, M., Rondepierre, B., Steiner, A., and van Zee, M. (2024). Flax: A neural network library and ecosystem for JAX.

Ho, J., Jain, A., and Abbeel, P. (2020). Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA. Curran Associates Inc.

Neftci, E. O., Mostafa, H., and Zenke, F. (2019). Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51-63.

# Learning with Restricted Boolean Models 

Mario Franco ${ }^{1 *}$ and Carlos Gershenson ${ }^{1}$<br>${ }^{1}$ School of Systems Science and Industrial Enginnering. Binghamton University, USA<br>*mfrancomndez@binghamton.edu


#### Abstract

Boolean networks are a popular model for describing biological systems, particularly genetic regulatory networks. Moreover, constructing a Boolean network that is in accordance with experimental observations can be a challenging and time-consuming task. Several efforts have been made to automatize the learning of an appropriate Boolean network model from experimental measurements (or a sequence of valid transitions). However, it is not yet clear whether it is possible to use Boolean networks as a general learning system. In particular, in this work we ask the question whether a Boolean network can actively and continuously learn from an online sequence of observations and losses/rewards. For this goal, we use a Restricted Boolean Network model that is more suitable for efficient inference and manipulation and show that this simple model is capable of online learning.


## Submission type: Late Breaking Abstracts

## Restricted Boolean Networks

Definition 1. A restricted Boolean network $(B N) \mathcal{B}$ is a binary-stateful directed weighted graph $\mathcal{G}$, with weights in $\{-1,0,1\}$, and $\forall i \in\{1, \cdots, n\}, f_{i}=\left(\sum_{j} w_{i j} \cdot x_{j}\right)>0$.

Informally, the previous definition simply says that all nodes can only interact with another node in two ways: inhibition or excitation (or don't interact at all). A node is $O N$ if and only if it receives more excitation than inhibition and at least one excitation signal.

We want to remark that this model is not novel, the previous definition is almost identical to a threshold network Zañudo et al. (2011); Tran et al. (2013) with only two differences: 1) threshold networks are real-valued and 2) the case when the sum is equal to zero is resolved by not changing the current state of the node. A similar model can also be found in Ouyang et al. (2014) and Kadelka and Hari (2025).

At first glance, the previous definition may look quite restrictive, and it is. Nonetheless, it can be shown that this restriction can be lifted with the help of an ancillary node.
Proposition 1. Boolean functions defined as above are not universal. However, they can be made universal be means of an ancillary node with its state fixed to $O N$.

Proof. First, it is easy to verify that it was not possible to construct the NOT function. Now, with the aid of an ancillary node we can express the $N O R$ function as $f= -x_{1}-x_{2}+1$. $\square$

Proposition 2. Every $B N \mathcal{B}$ can be simulated using an asynchronous restricted $B N$, extended as in proposition 1 , with a constant overhead.

Proof. Without loss of generality, assume that $f_{i} \in \mathcal{F}_{\mathcal{B}}$ is expressed in a canonical normal form, i.e., $f_{i}$ is composed entirely of $A N D s, O R s$ and $N O T s$. Now, these functions can be expressed in terms of $N O R$ as $\neg P=P \bar{\vee} P, P \vee Q=(P \bar{\nabla} Q) \bar{\nabla}(P \bar{\nabla} Q)$ and $P \wedge Q=(P \bar{\nabla} P) \bar{\nabla}(Q \bar{\nabla} Q)$. Therefore $f_{i}$ can be expressed in terms of $N O R$ with at most 3 times the original number of operators plus one auxiliary node. Observe that, in almost all cases, we will be required to perform an asynchronous update to perfectly mimic the dynamic of the original BN (similar to how computers operate: different operations may take different clock steps). For every step in $\mathcal{B}$, an additional of up to two steps in the restricted network are required in order to observe the same dynamic on the core nodes (the nodes that are in correspondence one-to-one with $\mathcal{B}$ ). Thus, $\mathcal{B}$ can be simulated with a restricted BN . $\square$

The previous propositions tell us that everything that can be model with a BN can also be model with a restricted BN .

## Learning with Restricted Boolean Model

Previous efforts have been made to automatically learn BN models, most of them focused on genetic algorithmic approaches (Park et al., 2023; Torres-Sosa et al., 2012). Rivera Torres et al. (2022) used a model checker to build a probabilistic Boolean network to control a power grid model. Another promising avenue is to use informationtheoretic approaches to find a minimal network that conforms to the specifications of the model (Sun et al., 2022). Similar to our work, Ouyang et al. (2014) trained a threshold BN to learn to mimic a known trajectory of a Boolean system using co-occurrence of genes in the trajectory as the guiding force.

In the present work, we study learning in a setup similar to what some living organisms have to confront: they receive a signal, produce an answer to that signal, and then receive a feedback (directly or indirectly). This setup is commonly referred to as online reinforcement learning. In particular, we are interested in the case in which learning occurs in a continuous manner and it is not possible to directly encode and save large pairs of inputs/outputs/rewards (e.g. due to limited resources).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-149.jpg?height=903&width=1785&top_left_y=260&top_left_x=141)
Figure 1: Learning summary of a restricted Boolean network. The network was trained to replicate the input patterns at the output position. Patterns are indicated with different colors for ease of exposition. It is important to remark that there are no direct connections between input and output nodes. (Left, Top) Error during learning. (Left, Bottom) Accuracy during learning. (Right, Top) Performance of the trained network without noise and (Right, Bottom) with noise.

Inspired by the similarities of this model with extremely simple neural models, we take an approach similar to predictive coding (Clark, 2013). In a nutshell, higher-level hierarchies try to predict the behavior of lower-level hierarchies and influence their future behavior. Thus, we decompose our Boolean model into a layered structure in which each layer can only talk to its immediate neighbors (up and below the hierarchy) and itself. Our learning algorithm is rather simple; first, we assume that we know the desired output state at the end of the network, then we use this state as the goal activity at that layer. Second, we run the network for a certain number of steps $T$. Each timestep, we update an activity trace (exponential moving average) of the state of $B_{i}$. Third, after the $T$ steps, we perform a learning update: At each layer, we compare the current activity trace with the target activity and update the kernel to make the target activity more likely given the activity of neighboring layers. This is achieved through a simple stochastic perturbation based on how much a node differs from its target activity.

As a proxy, we use a simple imitation game. A pattern is presented at the input of a layered BN and the goal is to recreate the same pattern at the output layer. In total, ten noiseless patterns are presented to the network randomly and continuously for 128 K timesteps. The BN is composed of 3 self-recurrent layers with connection to their immediate neighbors and is updated at regular intervals of 20 timesteps.

We assume that the output is correct if the majority of the pattern nodes are $O N$ and the nodes not in the pattern are $O F F$. A summary of the training of a single network is shown in figure 1. Manual inspection reveals that the BN network has robust and mostly correct behavior in both the absence and the presence of noise, with only 1 and 2 errors observed in a spawn of 25 k timesteps, respectively. Remarkably, training the network without any noise still produces a model that can handle a moderate amount of noise. Learning using deeper architectures is currently unstable; future refinements of the algorithm may be able to allow learning in deeper networks and more challenging contexts.

## Conclusion

In this work, we studied the possibility of using restricted BNs as general learning systems. We used a predictive-coding-like approach to learn to mimic patterns in a continuous setup using a hierarchical BN. The present learning algorithm is inefficient, but allows for learning in a recurrent setting.

Although currently limited, the present work suggests alternative strategies to the error gradient approach to learning in continuous settings using recurrent models. Further work in this direction could help us find efficient alternative strategies to backpropagation-based learning for neural networks.

## References

Clark, A. (2013). Whatever next? predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(3):181-204.

Kadelka, C. and Hari, K. (2025). Critical assessment of the ability of boolean threshold models to describe gene regulatory network dynamics. bioRxiv.

Ouyang, H., Fang, J., Shen, L., Dougherty, E. R., and Liu, W. (2014). Learning restricted boolean network model by timeseries data. EURASIP Journal on Bioinformatics and Systems Biology, 2014(1):10.

Park, K. H., Rozum, J. C., and Albert, R. (2023). From years to hours: Accelerating model refinement. bioRxiv.

Rivera Torres, P. J., Gershenson García, C., Sánchez Puig, M. F., and Kanaan Izquierdo, S. (2022). Reinforcement learning with probabilistic boolean network models of smart grid devices. Complexity, 2022:3652441.

Sun, J., AlMomani, A. A. R., and Bollt, E. (2022). Data-driven learning of boolean networks and functions by optimal causation entropy principle. Patterns, 3(11):100631.

Torres-Sosa, C., Huang, S., and Aldana, M. (2012). Criticality is an emergent property of genetic networks that exhibit evolvability. PLOS Computational Biology, 8(9):1-18 .

Tran, V., McCall, M. N., McMurray, H. R., and Almudevar, A. (2013). On the underlying assumptions of threshold boolean networks as a model for genetic regulatory network behavior. Front Genet, 4:263.

Zañudo, J. G. T., Aldana, M., and Martínez-Mekler, G. (2011). Boolean Threshold Networks: Virtues and Limitations for Biological Modeling, pages 113-151. Springer Berlin Heidelberg, Berlin, Heidelberg.

# A Challenging Visual Language Task for the Iterated Learning Model 

Hyoyeon Lee, Seth Bullock \& Conor Houghton<br>University of Bristol, UK, conor.houghton@bristol.ac.uk


#### Abstract

The iterated learning model simulates the transmission of language from generation to generation in order to explore how the constraints imposed by language transmission facilitate the emergence of language structure. Despite each modelled language learner starting from a blank slate, the presence of a language learning bottleneck limiting the number of utterances to which the learner is exposed can lead to the emergence of language that lacks ambiguity, is governed by grammatical rules, and is consistent over successive generations, that is, one that is expressive, compositional and stable, respectively. Here, for the first time, the semi-supervised iterated learning model has been applied to a language learning task involving the communication of complex visual meanings: seven-segment display images. Agents in this model are able to learn and transmit a language that is expressive: distinct codes are employed for all 128 glyphs; compositional: signal bits represent meaning components, and stable: the language does not change from generation to generation. ${ }^{1}$


Keywords iterated learning, compositionality, cultural evolution, autoencoder, inner speech

## Introduction

Synthetic, computational approaches to understanding the basic properties of language change are attractive since they offer an additional window on evolutionary linguistic phenomena (Christiansen and Kirby, 2003; Kirby, 2002). In particular, computational agent-based models of language learning can offer insight into the way that language learning constraints might influence, shape or determine language properties (Kirby, 2002). Here, one such model, the iterated learning model (ILM; Kirby and Hurford, 2002), which has been successfully applied to simple artificial language learning tasks in which meanings and signals are typically limited to short bit-strings, is extended to a more complex task in which the 'meanings' are glyphs based on the sevensegment LED displays that are often used on calculators, digital watches and digital clocks (Clark, 1929).

[^8]![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-152.jpg?height=243&width=759&top_left_y=824&top_left_x=1117)
Figure 1: Example images. Each base glyph is a $28 \times 28$ image based on the classic seven-segment LED display. All 128 glyphs are employed, many of which do not correspond to digits or letters. The three glyph types presented here correspond to digits 1,5 , and 8 . For each base glyph, 100 variant images were produced using small random rotations, translations and changes in intensity.

Ultimately our goal is to understand the relationship between language properties and cognitive processes and we believe the model reported here is a step towards this goal.

## Method and Results

The ILM is an agent-based simulation designed to explore conditions under which languages tend to become expressive, compositional and stable (Kirby, 2001; Kirby and Hurford, 2002). An ILM comprises a language-using 'tutor' agent that instructs an initially naïve 'pupil' agent by exposing it to a finite set of utterances, each pairing a meaning with the signal that expresses it in the tutor's language. After this period of instruction, the pupil becomes a tutor and teaches a new initially naïve pupil in the same fashion. A language learning bottleneck is a key feature of the model: since each pupil is only exposed to a finite subset of the tutor's language during learning, when it becomes a tutor itself it will need to generate signals for meanings that were not part of its learning experience. This demand for generalization is critical to the behaviour of the ILM (Kirby and Hurford, 2002).

There are several versions of the ILM, but a typical example due to Kirby and Hurford (2002) features a space of simple possible languages that each map from a finite set of bitstring meanings, $\mathcal{M}$, to a finite set of bitstring signals, $\mathcal{S}$. The agent has two maps, an encoder $e$ mapping meanings to signals and a decoder, mapping signals to meanings.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-153.jpg?height=373&width=1435&top_left_y=260&top_left_x=315)
Figure 2: The network is trained in four different configurations, as a supervised encoder mapping glyphs to signals, the grey rectangle denotes the discretization to zeros and one; as a supervised decoder, mapping words or 'thoughts' to glyphs and two types of unsupervised autoencoder. One maps a glyph to itself, representing the training of the primary visual system, while the other maps through the inner parts of the network and represents the internalization of language.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-153.jpg?height=364&width=1774&top_left_y=845&top_left_x=141)
Figure 3: An expressive, compositional and stable language evolves. Thin lines show measures of expressivity, $x$, compositionality, $c$, and stability, $s$ for ten individual 100 -generation instantiations with $n_{l}=7$. Thick lines show the average values.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-153.jpg?height=154&width=442&top_left_y=1361&top_left_x=351)

Here, meanings are 128 seven-segment glyphs, Fig. 1 and so the inputs are $28 \times 28$ images with mild random rotations, translations and intensity changes, while the signals are 7-bit strings (Clark, 1929). Following Bunyan et al. (2025) both $E$ and $D$ are neural networks; each is a feed-forward network, the encoder is a $784 \times 128 \times 7 \times 7 \times 7$ all-to-all network mapping from the glyphs to continuous length seven latent space vectors that can be considered to represent an agent's 'thoughts'. It is useful to consider the encoder as comprising two parts, the image encoder $E_{i}$, the $784 \times 128 \times 7$ network that reduces the dimension of the image, and the word encoder $E_{w}$, the $7 \times 7 \times 7$ map from the reduced representations to the latent space. The latent space 'thoughts' are mapped to words by a discretization map $\delta$. The decoder maps are similar, but map from the latent space back to the glyphs.

Crucially, for training to succeed the neural networks need to be trained in different configurations Fig. 2: as encoder, $E=E_{w} \circ E_{i}$ and decoder $D=D_{i} \circ D_{w}$, using examples provided by the tutor, and as two types of auto-encoder, the 'outer' autoencoder $D_{i} \circ E_{i}$ which maps the glyphs down to a seven-dimensional representation and then back to the original glyph and an 'inner' autoencoder $D_{w} \circ E_{w}$ that maps the reduced representations to the latent space and then back to a reduced representation. These two autoen-
coders represent training of the primary sensory pathways and the internalization of language (Meister et al., 1991; Shatz and Stryker, 1978; Cang and Feldheim, 2013; Vygotsky, 1986; Carruthers, 2002; Bunyan et al., 2025; Bullock and Houghton, 2024).

In our simulations, the language becomes expressive, compositional and stable (Fig. 3): each meaning bit aligns with one of the seven segments and reconstructions through $D \circ E$ are crisp. Noisier regimes require a larger latent space, the details of this are not show here.

## Discussion

The ILM is a very simple model but in its current semisupervised form, it supports a striking hypothesis about the origin of language: rather than being wholly concerned with encoding and decoding external utterances from a shared language, the agent must also rely on auto-encoder-driven learning of its own "language of thought" (Vygotsky, 1986; Carruthers, 2002). Clearly, there is a substantial gap between the simplicity of the model and the subtlety of the language of thought idea; to bridge this the model needs to be applied to more complex examples. This paper is a first step in this programme and comes with an interesting lesson, the same autoencoder mechanism that is useful for language learning and crucial for language evolution is also useful in learning the more peripheral parts of the pathway, analogous to primary visual processing (Meister et al., 1991; Shatz and Stryker, 1978; Cang and Feldheim, 2013).

## References

Bullock, S. and Houghton, C. (2024). Modeling language contact with the iterated learning model. In Artificial Life Conference Proceedings, volume 36, page 54.

Bunyan, J., Bullock, S., and Houghton, C. (2025). An iterated learning model of language change that mixes supervised and unsupervised learning. PLOS Complex Systems, 2(3):e0000030.

Cang, J. and Feldheim, D. A. (2013). Developmental mechanisms of topographic map formation and alignment. Annual Review of Neuroscience, 36:51-77.

Carruthers, P. (2002). The cognitive functions of language. Behavioral and Brain Sciences, 25(6):657-674.

Christiansen, M. H. and Kirby, S., editors (2003). Language Evolution. Oxford University Press, Oxford.

Clark, E. H. (1929). Evolution of the call-indicator system. Bell Laboratories Record, 8(5):171-173.

Kirby, S. (2001). Spontaneous evolution of linguistic structure: An iterated learning model of the emergence of regularity and irregularity. IEEE Transactions on Evolutionary Computation, 5(2):102-110.

Kirby, S. (2002). Learning, bottlenecks and the evolution of recursive syntax. In Briscoe, E., editor, Linguistic Evolution Through Language Acquisition: Formal and Computational Models, pages 173-204. Cambridge University Press, Cambridge, U.K.

Kirby, S. and Hurford, J. (2002). The emergence of linguistic structure: An overview of the iterated learning model. In Cangelosi, A. and Parisi, D., editors, Simulating the Evolution of Language, pages 121-148. Springer London.

Meister, M., Wong, R. O. L., Baylor, D. A., and Shatz, C. J. (1991). Synchronous bursts of action potentials in ganglion cells of the developing mammalian retina. Science, 252(5008):939943.

Shatz, C. J. and Stryker, M. P. (1978). Ocular dominance in layer iv of the cat's visual cortex and the effects of monocular deprivation. Journal of Physiology, 281:267-283.

Vygotsky, L. S. (1986). Thought and Language. MIT Press, Cambridge, MA, revised edition edition.

# Exploring the Landscape of Non-Equilibrium Memories with Neural Cellular Automata 

Ehsan Pajouheshgar ${ }^{1}$, Aditya Bhardwaj ${ }^{2}$, Nathaniel Selub ${ }^{3}$, and Ethan Lake ${ }^{3}$<br>${ }^{1}$ EPFL, Switzerland $\quad{ }^{2}$ Caltech, USA $\quad{ }^{3}$ Berkeley, USA<br>ehsan.pajouheshgar@epfl.ch a7b@caltech.edu nathanielselub@berkeley.edu elake@berkeley.edu


#### Abstract

We investigate the landscape of many-body memories: cellular automata that retain information about their initial states for thermodynamically long time scales, even in the presence of arbitrary perturbations. The only well-studied memory is Toom's rule, discovered in 1980. Using a combination of rigorous proofs and machine learning methods, we show that the landscape of many-body memories is in fact quite vast. We discover memories that correct errors in ways qualitatively distinct from Toom's rule, including a memory that preserves information only in the presence of noise. Taken together, our results show that physical systems can perform robust information storage in many distinct ways, and demonstrate that the physics of many-body memories is richer than previously realized. Interactive visualizations of the CAs studied in this work are available at https://memorynca.github.io/2D.


## Submission type: Late Breaking Abstracts

This work investigates non-equilibrium many-body memories in 2D spin systems. The dynamics we study are designed to remember one bit of information, encoded in the sign of the system's magnetization. We consider dynamics acting on spins $s_{\mathbf{r}} \in\{ \pm 1\}$ located on the sites of an $L \times L$ lattice with periodic boundary conditions. The dynamics is defined as a probabilistic cellular automaton (CA), specified by a noise strength $p \in[0,1]$, a noise bias $\eta \in[-1,1]$, and a CA rule $\mathcal{A}$. Letting $\mathcal{R}_{\mathbf{r}}$ denote the $3 \times 3$ block of sites centered at a site $\mathbf{r}$, the dynamics updates $s_{\mathbf{r}}$ according to a particular function $\mathcal{A}$ of the spins on $\mathcal{R}_{\mathbf{r}}$. When noise is included, the dynamics acts by sending $s_{\mathbf{r}} \mapsto \mathcal{A}\left(\left.s\right|_{\mathcal{R}_{\mathbf{r}}}\right)$ with probability $1-p$, and $s_{\mathbf{r}} \mapsto \pm 1$ with probability $p(1 \pm \eta) / 2$.

In the math and computer science literature, CA are normally updated synchronously: they are run in discrete time, and all sites are updated at once. From the physics perspective, an asynchronous update scheme is often more natural, as it corresponds to a continuous-time Markov process. We will consider both types of updates in this work.

We say that $\mathcal{A}$ defines a robust memory if there is an open ball in the $(p, \eta)$ plane within which the dynamics preserves the sign of the magnetization $m=L^{-2} \sum_{\mathbf{r}} s_{\mathbf{r}}$ for a time that diverges as $L \rightarrow \infty$. Having a robust memory is only possible if regions of errors are corrected in a time proportional to their linear size: this is because a bias $\eta$ favoring a minority domain causes it to ballistically expand, and this
expansion rapidly destroys the encoded information if the minority domain is not quickly eroded by the dynamics. In this context, the problem of constructing a robust memory is therefore equivalent to finding a CA that quickly identifies and corrects arbitrarily-shaped domains of errors.

The most well-known robust memory in 2D is Toom's rule, $\mathcal{A}_{\text {Toom }}\left(s_{\mathbf{r}}\right)=\operatorname{maj}\left(s_{\mathbf{r}}, s_{\mathbf{r}+\hat{\mathbf{x}}}, s_{\mathbf{r}+\hat{\mathbf{y}}}\right)$, where $\operatorname{maj}(S)$ takes the majority vote of the set $S$ (Toom, 1980). The triangular shape of the majority voting region ensures that minority domains (regardless of their shape) are ballistically eroded starting from their top-right corners. To date, all constructions of robust memories with local order parameters in the physics literature descend from this single mechanism (Ray et al., 2024; Machado et al., 2023; McGinley et al., 2022). It is then natural to ask: is Toom's rule the only way of achieving a robust 2D memory?
A first class of examples: We prove that the triangular geometry of Toom's rule is in fact not special, and that most ways of doing local majority voting yield robust memories under synchronous updates. As a concrete example, consider the CA rule A defined by the voting region $\mathcal{V}_{\mathbf{r}}=\mathbf{r}+\{\hat{\mathbf{y}}, \hat{\mathbf{x}}+\hat{\mathbf{y}}, \hat{\mathbf{x}},-\hat{\mathbf{y}},-\hat{\mathbf{x}}\}$. As illustrated in the top row of Fig. 1 a , A erodes minority domains by mapping them to a series of increasingly small pengaton-shaped regions.

Our theorem only applies to dynamics with synchronous updates. The extension to asynchronous updates is nontrivial, as there exist systems which retain memory only when updated synchronously (Lake et al., 2025). To establish results for asynchronous dynamics, we therefore turn to numerics. We use Monte Carlo simulations to estimate the logical failure rate, defined as the probability for a magnetization flip to occur by time $O(L)$ when the system is initialized in its least stable memory state:

$$
p_{\text {log }}=\max _{s_{ \pm}} \mathrm{P}\left[\operatorname{maj}\left(\mathcal{A}^{c L}(s)\right) \neq \operatorname{maj}(s)\right],
$$

where $s_{ \pm}$are respectively the all-1s and all-(-1)s states, and $c$ is a constant, fixed at $c=5$ in our numerics. For any memory we must have $\lim _{L \rightarrow \infty} p_{\text {log }}=0$, and for the CA under consideration we expect, for small $p$, the scaling

$$
p_{\log } \sim\left(p / p_{c}\right)^{\lambda L}
$$

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-157.jpg?height=597&width=1548&top_left_y=283&top_left_x=255)
Figure 1: Correction of large minority domains of 1 spins (black) for different memories, with the noise bias $\eta=+1$ chosen to maximally favor the error. Time runs left to right. b) A schematic phase diagram in the ( $p, \eta$ ) plane for typical rules found with our NCA architecture. c) Schematic phase diagram of the D rule, with a re-entrant noise-stabilized memory phase; the tip of the memory phase is around $(p, \eta) \approx(0.4,0.08)$. d) $p_{\text {log }}$ against $p$ for asynchronous updates, shown for the A automaton at $\eta=0$. The dashed lines are fits to $p_{\text {log }} \propto p^{\lambda L}$ with $\lambda \approx 1.3$. e) the same as d , but for the C automaton at $\eta=-1$; here $\lambda \approx 2.6$.

for constants $p_{c}, \lambda$. The logical failure rate $p_{\text {log }}$ for A dy namics with asynchronous updates and bias $\eta=0$ is plotted in Fig. 1 d, where we obser ve a threshold crossing at $p_{c} \approx 26 \%$-significantly higher than the $p_{c, \text { Toom }} \sim 14 \%$ of Toom's rule (Bennett and Grinstein, 1985; Ray et al., 2024).
Neural Cellular Automata: Do there exist robust memories beyond asymmetric majority voters? To efficiently explore the space of possible dynamics, we use a machine-learning-guided approach inspired by the neural cellular automaton (NCA) framework (Mordvintsev et al., 2020). In this setting, $\mathcal{A}$ is initially a random continuous-state CA rule (operating on $\mathbb{R}$-valued spins), parameterized by a small neural network. The neural network is optimized using gradient descent to minimize a loss function that is designed to measure $\mathcal{A}$ 's ability to erase large domains of errors in the presence of noise. Including large error regions in the training data, instead of relying on their spontaneous occurrence from noisy dynamics, enables faster and more targeted learning. After a fixed number of optimization steps, we discretize $\mathcal{A}$ to $\mathbb{Z}_{2}$-valued spins.

In a set of 1000 training runs with different random seeds, 37 CA converged to robust memories, none of which implemented majority votes. In most cases where the learned rule was not a memory, the dynamics was observed to perform something similar to a $C_{\pi}$-symmetric majority vote. Finally, while training was performed only with synchronous updates, all of the learned memory rules were observed to remain robust after making the updates asynchronous.

Representative examples of learned memories are shown in the last three rows of Fig. 1 a. The automaton B in the second row erodes domains by shrinking them into small triangles, while C in the third row squeezes domains along an oblique angle. Both $\mathrm{B}, \mathrm{C}$ are monotonic, and possess no symmetries. These rules have phase diagrams and a scal-
ing of $p_{\text {log }}$ qualitatively similar to that of A . The automaton D shown in the fourth row of Fig. 1 a, which is one of the non-monotonic rules found in training, is rather unusual. At $p=0$ it has exponentially many absorbing states, created by inserting small islands of -1 s in a background of 1 s (in the language of (Rakovszky et al., 2024; Gacs, 2017), it is thus not a linear eroder). Under unbiased noise these islands percolate, resulting in a unique steady state dominated by -1 s . With a modest amount of noise biased to favor 1 , however, the noisy CA dynamics render the islands unstable. This results in the phase diagram of Fig. 1 c , featuring a "re-entrant" memory phase in an intermediate range of $p$. The D automaton is thus a noise-stabilized memory, and its existence demonstrates that even non-eroding automata can define robust memories.

Outlook: The systems studied in this work are capable of "homeostasis": the robust protection of (here, a single bit of) information against degradation by noise. It is natural to wonder about mechanisms by which richer behavior, like nontrivial information processing and environmental adaptation, can robustly emerge in locally-interacting systems. It is well known that simple CA rules can display remarkably life-like emergent behavior (Langton, 1997; Chan, 2018; Mordvintsev et al., 2020), but in none of these cases has the problem of noise resilience been carefully addressed, and an outstanding problem is to identify the complexity threshold that must be crossed in order for CA to exhibit complex and robust emergent behavior. The constructions of Gacs (Gács, 2001, 1989)-whose design calls to mind the genetic code of an elaborate artificial organism-provide extremely complicated starting points, but there is clearly much more to be done. We expect the NCA approach developed in the present work to be useful in this regard.

## Acknowledgements

This research was supported by a Miller Research Fellowship and the National Science Foundation Graduate Research Fellowship.

## References

Bennett, C. H. and Grinstein, G. (1985). Role of irreversibility in stabilizing complex and nonergodic behavior in locally interacting discrete systems. Physical review letters, 55(7):657.

Chan, B. W.-C. (2018). Lenia-biology of artificial life. arXiv preprint arXiv:1812.05433.

Gács, P. (1989). Self-correcting two-dimensional arrays. Adv. Comput. Res., 5:223-326.

Gács, P. (2001). Reliable cellular automata with self-organization. Journal of Statistical Physics, 103:45-267.

Gacs, P. (2017). Eroders.
Lake, E. et al. (2025). Squeezing codes. To appear.
Langton, C. G., editor (1997). Artificial Life: An Overview. MIT Press, Cambridge, MA.

Machado, F., Zhuang, Q., Yao, N. Y., and Zaletel, M. P. (2023). Absolutely stable time crystals at finite temperature. Physical Review Letters, 131(18):180402.

McGinley, M., Roy, S., and Parameswaran, S. (2022). Absolutely stable spatiotemporal order in noisy quantum systems. Physical Review Letters, 129(9):090404.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing neural cellular automata. Distill, 5(2):e23.

Rakovszky, T., Gopalakrishnan, S., and Von Keyserlingk, C. (2024). Defining stable phases of open quantum systems. Physical Review X, 14(4):041031.

Ray, A., Laflamme, R., and Kubica, A. (2024). Protecting information via probabilistic cellular automata. Physical Review $E$, 109(4):044141.

Toom, A. L. (1980). Stable and attractive trajectories in multicomponent systems. Multicomponent random systems, 6:549575.

# Costly information and its role in reducing threshold uncertainty in collective risk dilemmas 

Genki Ichinose ${ }^{1}$, Mizuki Yokoyama ${ }^{1}$, Wakaba Tateishi ${ }^{2}$, Hiromu Ito ${ }^{3}$, Tom Lenaerts ${ }^{4,5}$, and Elias Fernández Domingos ${ }^{4,5}$<br>¹Department of Mathematical and Systems Engineering, Shizuoka University, Japan<br>${ }^{2}$ Faculty of Business Administration, Hokkaido Musashi Women's University, Japan<br>${ }^{3}$ Department of International Health and Medical Anthropology, Institute of Tropical Medicine, Nagasaki University, Japan<br>${ }^{4}$ Artificial Intelligence lab, Computer Science Department, Vrije Universiteit Brussel, Belgium<br>${ }^{5}$ Machine Learning Group, Computer Science Department, Univeristé Libre de Bruxelles<br>ichinose.genki@shizuoka.ac.jp


#### Abstract

Collective risk dilemmas (CRDs) describe situations in which individuals must cooperate to avoid collective losses, such as those arising from climate change or resource overuse. Previous experimental and theoretical studies have shown that uncertainty about the exact contribution threshold significantly undermines cooperation. Here, we investigate whether costly information acquisition can mitigate this effect. We conducted an online pilot experiment where participants faced a CRD with uncertain thresholds. In the uncertainty treatment, the threshold was unknown, while in the information treatment, participants could pay to privately learn it. About $65 \%$ of participants chose to pay, and the information improved success rates. However, individual contributions were lower as participants optimised to contribute just enough to reach the threshold. These findings highlight that while the availability of costly information has an overall positive effect on group success, the information asymmetries it creates can lead to more economic inequality.


## Submission type: Late Breaking Abstracts

## Introduction

Social dilemmas are situations where individual and collective interests conflict, often leading to suboptimal outcomes for all parties involved. Many real social dilemmas face nonlinear outcomes, where participants risk extreme losses if they do not cooperate to reach a common threshold. The collective risk dilemma (CRD) operationalises these situations through a game-theoretic framework (Milinski et al., 2008).

Previous experimental studies have shown that threshold uncertainty, when the exact contribution level required to avoid losses is unknown, significantly undermines cooperation. Dannenberg et al. (2015) found that when not only the threshold value is uncertain, but also the distribution from which it is drawn is unknown (ambiguous thresholds), participants' contributions decrease even further.

While uncertainty and ambiguity over collective threshold may cause a collapse of collective-action, in many realworld collective risk dilemmas, such as environmental conservation and resource management (Hardin, 1968), public health (Fine and Clarkson, 1986), disaster prevention (Kunreuther and Pauly, 2004; Kunreuther, 1986), and mitigation of climate change (Ostrom, 2010; Tavoni et al., 2011), stakeholders can pay costs to reduce uncertainty.

Here, we investigate through a behavioural experiment in the CRD, the impact that the availability of costly information has on individual and collective contributions in the presence of threshold uncertainty

## Methods

We conducted an online experiment using the CRD with threshold uncertainty Dannenberg et al. (2015). In this game, participants receive an initial endowment of 50 experimental currency units (ECUs) and, in each round, decide how much ( 0,2 , or 4 ECUs) to contribute to a public account.

If the total group contribution across 10 rounds reaches or exceeds a pre-specified threshold, the group succeeds in avoiding the collective loss, and all participants keep their remaining endowment. If the total contribution is below the threshold, all participants lose $90 \%$ of their remaining endowment and keep only the remaining $10 \%$.

In addition, before round 1 and 6, participants can make non-binding contribution proposals (following Dannenberg et al. (2015)). Specifically, each participant is asked how much they would like their group to contribute in total to the public account (between 0 and 160 ECUs). The proposal is displayed to the other group members during the game, but it is non-binding.

Note that in the original CRD game, the threshold is known to all participants (Milinski et al., 2008). In contrast, following Dannenberg et al. (2015), we assume that the threshold is unknown to participants. In our setting, the threshold is drawn from a uniform distribution within the
range of [0,160] in steps of 20, $(T \sim U[0 . .20 . .160])$.
We run an online pilot experiment with one factor and two levels: no costly information/costly information. In the Uncertainty control treatment, participants did not know the exact threshold (no costly information), which was drawn from the uniform distribution described above. In the Uncertainty + Private information treatment, conducted under the same rules, participants could decide before round 1 and 6 whether to pay 10 ECU to learn the exact threshold for their group (costly information). This information was private, meaning that only those who purchased it could see the threshold, and other participants did not know who purchased the information. However, since participants could pledge how much the group should contribute in total to the public account before rounds 1 and 6, they could use this opportunity to share the learned threshold value with the group. Nevertheless, participants could not share whether or not they purchased information.

Participants were recruited via Yahoo!Crowdsourcing in Japan. For each treatment, 100 participants were initially recruited. The Uncertainty treatment was conducted on July 24, 2025, with 95 participants successfully completing the experiment. The Uncertainty + Private information treatment was conducted on August 7, 2025, with 84 participants completing the experiment. In both cases, the remaining participants either failed the comprehension test or dropped out during the game.

## Results

We compare group performance and individual contributions between the two treatments. In the Uncertainty + Private information treatment, 55 participants purchased information ( 48 in round 1 and 7 in round 6 ), which means that about $65 \%$ of the participants chose to pay the cost in order to learn the threshold.

The success rate, which is the proportion of groups reaching the contribution threshold, is $\sim 10 \%$ higher in the Uncertainty + Private information treatment than in the Uncertainty treatment (Fig. 1). This indicates that access to information, even at a cost, improves collective success.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-161.jpg?height=428&width=570&top_left_y=2005&top_left_x=287)
Figure 1: Success rates of groups in the two treatments.

Yet, we also observe a variation in individual contributions. The average total individual contribution was lower in the Uncertainty + Private information treatment (19 ECU) compared to the Uncertainty treatment (28 ECU) (Fig. 2). A plausible explanation is that, although the success rate was higher, individual contributions were lower because once participants could access the exact threshold, participants optimised to contribute just enough to reach the threshold rather than over-contributing as in the Uncertainty treatment.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-161.jpg?height=334&width=844&top_left_y=678&top_left_x=1069)
Figure 2: Individual total contributions in the two treatments.

Finally, as a consequence of the previous result, we find that groups in the Uncertainty treatment frequently contributed well above the threshold (mean $=111.5$ ), whereas groups in the Uncertainty + Private information treatment contributed amounts closer to the threshold (mean $=75.4$ ) (Fig. 3). This further supports that the individual optimisation hypothesis extends to group behaviour.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-161.jpg?height=346&width=846&top_left_y=1484&top_left_x=1069)
Figure 3: Final group contributions compared to the threshold (target) in the two treatments.

## Conclusion

This study examined how costly information acquisition influences cooperation in a collective risk social dilemma with threshold uncertainty. We conducted an online experiment in which participants could choose to pay for private information about the contribution threshold.

The results show that information improves the likelihood of group success but also reduces individual contributions, leading groups to contribute to optimise and just enough to reach the threshold. These findings highlight a trade-off: reducing uncertainty supports collective success, yet it limits the extra contributions that often emerge under uncertainty.

## References

Dannenberg, A. ad Löschel, A., Paolacci, G., Reif, C., and Tavoni, A. (2015). On the provision of public goods with probabilistic and ambiguous thresholds. Environmental and Resource Economics, 61:365-383.

Fine, P. and Clarkson, J. (1986). Individual versus public priorities in the determination of optimal vaccination policies. American Journal of Epidemiology, 124:1012-1020.

Hardin, G. (1968). The tragedy of the commons. Science, 162:1243-1248.

Kunreuther, H. (1986). Disaster mitigation and insurance: Learning from katrina. The Annals of the American Academy of Political and Social Science, 124:1012-1020.

Kunreuther, H. and Pauly, M. (2004). Neglecting disaster: Why people do not insure against large losses. Journal of Risk and Uncertainty, 28:5-21.

Milinski, M., Sommerfeld, R., Krambeck, H., Reed, F., and Marotzke, J. (2008). The collective-risk social dilemma and the prevention of simulated dangerous climate change. Proceedings of the National Academy of Sciences, 105:22912294.

Ostrom, E. (2010). Polycentric systems for coping with collective action and global environmental change. Global Environmental Change, 20:550-557.

Tavoni, A., Dannenberg, A., Kallis, G., and Löschel, A. (2011). Inequality, communication, and the avoidance of disastrous climate change in a public goods game. Proceedings of the National Academy of Sciences, 108:11825-11829.

# Computational Evolutionary Psychiatry Account for Evolutionary Paradox in Autism Spectrum Disorder 

Shumpei Kurosaki ${ }^{1}$, Takafumi Soda ${ }^{1}$, Hiroki Kojima ${ }^{1}$, and Yuichi Yamashita ${ }^{1}$<br>${ }^{1}$ National Center of Neurology and Psychiatry, Japan kurosaki.s@ncnp.go.jp


#### Abstract

Autism spectrum disorder (ASD) is highly heritable and relatively common despite reduced reproductive fitness, presenting an evolutionary paradox. Genomic studies link ASD risk to intelligence-related genes and signals of positive selection, but the neural and cognitive mechanisms remain unclear. We addressed this using neuroevolution, integrating genes, neural circuits, cognition, and natural selection within a simulation-based framework. Artificial genomes encoded parameters of excitatory-inhibitory recurrent neural networks (E/I-RNNs) modeling prefrontal circuits. Networks were trained on a delayed matching-to-sample (DMS) task, with maximum delay as an index of intellectual ability, and selected through genetic algorithms. ASD-like traits were modeled as reduced cognitive flexibility in a rule-switching variant of the DMS task. Across 100 generations, many networks evolved good memory performance, while a persistent subpopulation showed impaired flexibility. Parameter analyses identified excitatory and inhibitory time constants as key determinants. Notably, certain parameter combinations that improved memory task performance were associated with impaired cognitive flexibility. These results suggest ASD traits may persist as byproducts of selection for higher cognition.


## Submission type: Late Breaking Abstracts

## Introduction

Autism spectrum disorder (ASD) is characterized by high heritability and low marriage rates, yet its prevalence remains relatively high at $1-5 \%$ (Maenner et al., 2020; Sasayama et al., 2021). This observation appears inconsistent with the principle of natural selection, which predicts that heritable traits not contributing to reproductive success should decline in frequency. Such paradoxes lie at the core of evolutionary psychiatry, which applies insights from evolutionary biology to address the question of why vulnerability to certain psychiatric disorders persists (Nesse, 2023). With recent advances in genomic technology, overlaps between intelligence-related genes and ASD risk genes have been reported, as well as evidence of positive selection on some of these loci (Clarke et al., 2016; Hagenaars et al., 2016). These findings suggest that traits underlying higher cognitive functions may have undergone positive selection, with vulnerability to ASDlike phenotypes emerging as a byproduct (Crespi, 2016). However, conventional research has remained at the genome-level analyses, leaving unresolved a critical question: which neural circuits and cognitive traits have
been shaped by natural selection to increase susceptibility to ASD-like phenotypes?

To overcome these limitations, simulation-based approaches rooted in artificial life and evolutionary algorithms have been explored. Among them, neuroevolution provides a framework for modeling the evolution of neural circuits, and its applications to neuroscience have grown in recent years (Miikkulainen, 2025). In this study, we extend this approach to evolutionary psychiatry, applying neuroevolution to test hypotheses about the persistence of vulnerability to ASD.

## Method

We aimed to develop a multi-layered model integrating genes, neural circuits, cognition, and natural selection to explore mechanisms maintaining ASD-like traits in populations (Figure 1). We designed an individual model endowed with various parameters encoded as an artificial genome (Thomas et al., 2016). Each individual was evaluated on a cognitive task, and genetic algorithm was applied in which task performance determined selection and inheritance of the artificial genome.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-164.jpg?height=609&width=822&top_left_y=1637&top_left_x=1091)
Figure 1: Overview of evolutionary simulation.

As the individual model, we employed an excitatoryinhibitory recurrent neural network (E/I-RNN) that mimics the neural circuit of the executive functions including the prefrontal cortex (Song et al., 2016). Hyperparameters representing properties of neural circuit function and learning-such as the number of excitatory and inhibitory
units, time constants, and pruning rates-were encoded in the artificial genome. Cognitive performance was assessed using a delayed matching-to-sample (DMS) task (Kim \& Sejnowski, 2021; Masse et al., 2019). In the simulations, E/I-RNNs generated from the artificial genome were trained on the DMS task, and the maximum delay interval for which correct responses were achieved served as an index of intellectual ability. This maximum delay was used as the fitness measure, and genetic algorithms were employed for genome selection and generational turnover.

As an ASD-related trait, we focused on difficulties in cognitive flexibility (Idei et al., 2018; Poljac \& Bekkering, 2012; Soda et al., 2023). Performance on a rule switching task derived from DMS task requiring cognitive flexibility was defined as an index of ASD-like characteristics. During the evolutionary process in which intellectual ability was used as the fitness criterion, we examined whether individuals with ASD-like traits would emerge and whether their frequency could be maintained.

## Result

Evolutionary simulation over 100 generations demonstrated that many individual networks acquired the ability to solve DMS tasks with long delay intervals, indicating successful simulation of evolution when intellectual ability was used as the fitness measure (Figure 2). At the same time, individuals capable of solving longdelay DMS tasks also tended to perform well on the rule switching task. Nevertheless, even after repeated generational turnover, individuals with impaired ruleswitching ability persisted within the population (Figure 3).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-165.jpg?height=419&width=810&top_left_y=1672&top_left_x=157)
Figure 2: Changes in fitness across generations.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-165.jpg?height=383&width=396&top_left_y=2174&top_left_x=164)
![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-165.jpg?height=380&width=401&top_left_y=2177&top_left_x=566)

Figure 3: Scatter plots of maximum delay interval (fitness) and rule-switching performance (flexibility). A subset of individuals with low cognitive flexibility (flexibility index $<0.6$ ) appeared and was maintained across generations.

To identify genome-level mechanisms underlying the persistence of individuals with poor rule-switching performance, we conducted a parameter search experiment by randomly sampling individuals from the genetic parameter space. Using Freedman-Lane permutation test (Freedman \& Lane, 1983), we analyzed the main effects and interactions of each genomic parameter on both maximum delay interval and rule-switching task performance. The analysis revealed that the time constants of excitatory and inhibitory neurons, as well as their interaction, were major determinants of task outcomes. For maximum delay interval, interaction effects were particularly pronounced: when inhibitory neuron time constants were large, smaller excitatory neuron time constants were advantageous (Figure 4). By contrast, in the rule-switching task, smaller excitatory neuron time constants were generally associated with poorer performance, with this effect being most pronounced under conditions of low inhibitory neuron time constants. These findings suggest that task-dependent differences in the interaction effects of the same neural parameters may underlie the mechanism by which individuals with low rule-switching performance emerge and are maintained, even as baseline DMS performance improves.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-165.jpg?height=391&width=817&top_left_y=1512&top_left_x=1098)
Figure 4: The interaction of excitatory and inhibitory time constants ( $\tau_{\text {exc }}, \tau_{\text {inh }}$ ) on fitness and flexibility.

## Discussion

Our simulations suggest that ASD-like traits can persist under strong selection for intellectual ability. This persistence may arise from neural time constant interactions that support memory but impair cognitive flexibility. These findings point to a potential circuit-level mechanism by which vulnerability to ASD-like phenotypes is maintained as a byproduct of selection for higher cognition. Although limited to E/I-RNNs and DMS tasks, our approach illustrates how evolutionary simulations can complement genomic studies in evolutionary psychiatry by linking genetic parameters to circuit dynamics and pleiotropic cognitive effects.

## Acknowledgements

The authors declare no conflicts of interest. This research was partly supported by JSPS KAKENHI JP25H01173 and JP24H00076, JST CREST JPMJCR21P4, JST Moonshot R\&D JPMJMS2031, AMED Brain/MINDS2.0 JP24wm0625407, and Intramural Research Grant (6-9, 79) for Neurological and Psychiatric Disorders of NCNP.

## References

Clarke, T.-K., Lupton, M. K., Fernandez-Pujals, A. M., Starr, J., Davies, G., Cox, S., Pattie, A., Liewald, D. C., Hall, L. S., MacIntyre, D. J., Smith, B. H., Hocking, L. J., Padmanabhan, S., Thomson, P. A., Hayward, C., Hansell, N. K., Montgomery, G. W., Medland, S. E., Martin, N. G., ... McIntosh, A. M. (2016). Common polygenic risk for autism spectrum disorder (ASD) is associated with cognitive ability in the general population. Molecular Psychiatry, 21(3), 419-425.

Crespi, B. J. (2016). Autism as a disorder of high intelligence. Frontiers in Neuroscience, 10, 300.

Freedman, D., \& Lane, D. (1983). A nonstochastic interpretation of reported significance levels. Journal of Business \& Economic Statistics: A Publication of the American Statistical Association, 1(4), 292.

Hagenaars, S. P., Harris, S. E., Davies, G., Hill, W. D., Liewald, D. C. M., Ritchie, S. J., Marioni, R. E., Fawns-Ritchie, C., Cullen, B., Malik, R., METASTROKE Consortium, International Consortium for Blood Pressure GWAS, SpiroMeta Consortium, CHARGE Consortium Pulmonary Group, CHARGE Consortium Aging and Longevity Group, Worrall, B. B., Sudlow, C. L. M., Wardlaw, J. M., Gallacher, J., Pell, J., McIntosh, A. M., ... Deary, I. J. (2016). Shared genetic aetiology between cognitive functions and physical and mental health in UK Biobank $(\mathrm{N}=112$ 151) and 24 GWAS consortia. Molecular Psychiatry, 21(11), 1624-1632.

Idei, H., Murata, S., Chen, Y., Yamashita, Y., Tani, J., \& Ogata, T. (2018). A neurorobotics simulation of autistic behavior induced by unusual sensory precision. Computational Psychiatry (Cambridge, Mass.), 2(0), 164-182.

Kim, R., \& Sejnowski, T. J. (2021). Strong inhibitory signaling underlies stable temporal dynamics and working
memory in spiking neural networks. Nature Neuroscience, 24(1), 129-139.

Maenner, M. J., Shaw, K. A., Baio, J., EdS1, Washington, A., Patrick, M., DiRienzo, M., Christensen, D. L., Wiggins, L. D., Pettygrove, S., Andrews, J. G., Lopez, M., Hudson, A., Baroud, T., Schwenk, Y., White, T., Rosenberg, C. R., Lee, L.-C., Harrington, R. A., ... Dietz, P. M. (2020). Prevalence of Autism spectrum disorder among children aged 8 years - Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2016. MMWR Surveillance Summaries, 69(4), 1-12.

Masse, N. Y., Yang, G. R., Song, H. F., Wang, X.-J., \& Freedman, D. J. (2019). Circuit mechanisms for the maintenance and manipulation of information in working memory. Nature Neuroscience, 22(7), 1159-1167.

Miikkulainen, R. (2025). Neuroevolution insights into biological neural computation. Science (New York, N.Y.), 387(6735), eadp7478.

Nesse, R. M. (2023). Evolutionary psychiatry: foundations, progress and challenges. World Psychiatry: Official Journal of the World Psychiatric Association , 22(2), 177-202.

Poljac, E., \& Bekkering, H. (2012). A review of intentional and cognitive control in autism. Frontiers in Psychology, 3, 436.

Sasayama, D., Kuge, R., Toibana, Y., \& Honda, H. (2021). Trends in autism spectrum disorder diagnoses in japan, 2009 to 2019. JAMA Network Open, 4(5), e219234.

Soda, T., Ahmadi, A., Tani, J., Honda, M., Hanakawa, T., \& Yamashita, Y. (2023). Simulating developmental diversity: Impact of neural stochasticity on atypical flexibility and hierarchy. Frontiers in Psychiatry, 14, 1080668.

Song, H. F., Yang, G. R., \& Wang, X.-J. (2016). Training excitatory-inhibitory recurrent neural networks for cognitive tasks: A simple and flexible framework. PLoS Computational Biology, 12(2), e1004792.

Thomas, M. S. C., Forrester, N. A., \& Ronald, A. (2016). Multiscale modeling of gene-behavior associations in an artificial neural network model of cognitive development. Cognitive Science, 40(1), 51-99.

# State Transitions Unlock Temporal Memory in Swarm-Based Reservoir Computing 

Tanner Lund ${ }^{1,2}$, Alyssa Adams ${ }^{2}$, Nathanael Aubert-Kato ${ }^{3}$, and Takashi Ikegami ${ }^{1}$<br>${ }^{1}$ The University of Tokyo, Japan<br>${ }^{2}$ Cross Labs, Kyoto, Japan<br>${ }^{3}$ Ochanomizu University, Tokyo, Japan<br>Correspondence: lund @ sacral.c.u-tokyo.ac.jp


#### Abstract

Swarms of simple agents provide a physical reservoir substrate: local interactions with motion-induced rewiring. We revisit swarm-based Reservoir Computing (RC) to clarify when swarms exhibit temporal memory and how that capacity scales. Our core result is empirical and reproducible: adding a second internal state with rule-based transitions (dispersed ↔ clustered) unlocks substantial memory and yields a linear scaling law with population size. Using a standardized, pure memory-capacity (MC) protocol (i.i.d. input, linear readout, no polynomial terms), we show that two-state swarms achieve high MC that increases with the number of agents $N$, while single-state swarms remain effectively memoryless under the same protocol. We provide a GPU-ready recipe and validate that the effect is intrinsic to swarm dynamics, not post-processing. We also contextualize the advance by standardizing evaluation and providing a clean, reproducible scaling characterization beyond prior swarm RC reports.


## Submission type: Late Breaking Abstracts

## Introduction

Reservoir Computing (RC) is originally a machine learning technique introduced to deal with Recurrent Neural Networks, highly non-linear where typical training approaches were impractical. In RC, the weights of the input layer and neural network itself (reservoir) are kept fixed, while the training is only applied to the last layer (called readout or output layer) (Jaeger, 2001; Lukoševičius and Jaeger, 2009). Eventually, that approach was extended to a large array of non-linear systems, including physical systems, where "weights" are either hard to define or difficult to modify (Tanaka et al., 2019). Among such non-linear systems, swarms provide an interesting substrate to implement reservoirs, as they are large dynamic structures emerging from local interactions (Bonabeau et al., 1999; Camazine et al., 2001). In particular, recent work have shown promising performance (Lymburn et al., 2021; Uehara et al., 2024). In this work, we attempt to further characterize swarm performance for RC in terms of Memory Capacity (MC), a typical benchmark in the field (Dale et al., 2019; Wringe et al., 2025).

In particular, we focus on two elements: (1) the impact of swarm size and (2) the impact of internal states in the individuals.

## Methods

We rely on simulations of 2D boid-like agents with alignment, cohesion, separation (Reynolds, 1987). We further use inertial velocity mixing and soft speed normalization to keep the speed of boids within usable range. The environment is a torus, to prevent both the swarm from moving without bound and having to set strong boundaries.

As a way to extend the capabilities of the swarm, we also define a binary internal state, with two modes (dispersed/clustered) with hysteretic transitions (local kineticenergy proxy). The goal is to introduce slow variables and metastable structures. A global input (dubbed "temperature") modulates the target speed at each time step.

The reservoir state is measured via a set of permutation/rotation/translation-invariant statistics: total area of the swarm, mean pairwise distance between neighbors, mean unit-velocity alignment, mean speed, speed standard deviation, angular momentum, clustered fraction, and mean speed per state (in the multistate case). Those features are whitened prior to readout. We use a linear ridge regression (L2) to decode delayed inputs under the canonical MC procedure (i.i.d. input; washout $\approx 500$; delays $1-50$ ). For the pure-MC condition, no polynomial terms are used. Scaling runs: two-state, $N \in\{800,1200,1600,2000\}, \sigma \in\{0,0.05,0.1\}$, five seeds per noise ( $n=15$ per $N$ ). Ablation runs (pure MC): single-state vs two-state at $N \in\{800,1200,1600\}$ with $n=10$ per $N$.

## Results

Linear MC scaling with population (two-state). MC $\approx 0.0123 \cdot N+1.61$ across $N=800-2000$; slope $95 \%$ CI [0.0122, 0.0123]. Per- $N$ means $\pm 95 \%$ CI $(n=15): 800: 11.41 \pm 0.08 ; 1200: 16.36 \pm 0.05 ; 1600: 21.22 \pm 0.05$; 2000: $26.14 \pm 0.06$ (Figure 1, Top).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-169.jpg?height=1054&width=817&top_left_y=283&top_left_x=166)
Figure 1: Memory capacity vs $N$ with transitions. Top: main scaling with fit and slope CI. Bottom: pure-MC slope confirmation.

Pure-MC slope confirmation (linear readout; no polynomial terms). Finalized subsets at $N=800,1600(n=10$ each) confirm MC $\approx 0.0122 \cdot N+1.64$. Slope is estimated on per- $N$ means; 95\% CIs are obtained by bootstrap across runs (Figure 1, Bottom).
Ablation at fixed $N$ (pure MC; means $\pm \mathbf{9 5 \%} \mathbf{C I} ; n=10)$. 800: single-state $0.11 \pm 0.02$ vs two-state $11.43 \pm 0.11(\sim 104 \times$ ). 1200 : $0.07 \pm 0.01$ vs $16.33 \pm 0.07(\sim 233 \times)$. 1600 : $0.11 \pm 0.01$ vs $21.19 \pm 0.05(\sim 193 \times)$ (Figure 2).
Short-delay and task context. As a compact check, $R^{2}$ at delays $1-4$ increases with $N$ in pure-MC runs. For MackeyGlass (open-loop), single-state achieves strong one-step prediction ( $R^{2} \approx 0.96$ ) while two-state performs poorly under our linear readout, consistent with the memory-prediction trade-off.

## Significance and limitations

Two-state transitions create slow variables (e.g., clustered fraction, interface persistence) that retain recent history, enabling temporal integration under a linear readout. Hysteresis induces dwell-time asymmetry, so state fractions evolve on a slower timescale than velocities, effectively integrat-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-169.jpg?height=415&width=774&top_left_y=296&top_left_x=1107)
Figure 2: Ablation at fixed $N$ under pure MC (linear readout; no polynomial terms). Bars show total MC (mean ± $95 \%$ CI; per-panel $n=10)$. Panels: $N=800,1200,1600$.

ing recent input fluctuations via state-coupled interaction gains. This yields predictable memory scaling in a physically embodied, dynamically connected substrate; as $N$ increases, the number of independent slow degrees of freedom grows, yielding approximately linear MC vs $N$; and provides a simple, GPU-ready recipe. We do not claim taskwise superiority: under our protocol, single-state swarms excel at instantaneous chaotic prediction (Mackey-Glass), while two-state swarms trade one-step prediction for temporal memory. Theoretical capacity bounds remain respected ( $\mathrm{MC} \leq N$ ), and our estimates use per- $N$ means with bootstrap CIs. Limitations include occasional metastability (absorbing-state collapse), restricted pure-MC confirmation at two $N$ values (slope nonetheless matches), and focus on a single benchmark task.

## Acknowledgments

This work is supported by the ALife Japan Future Talent Program 2024-2025 Fellowship and JSPS KAKENHI Grant Number JP24H00707. NAK is supported by JSPS KAKENHI Grant Number JP25H00608.

## References

Bonabeau, E., Dorigo, M., and Theraulaz, G. (1999). Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, New York, NY.

Camazine, S., Deneubourg, J.-L., Franks, N. R., Sneyd, J., Theraulaz, G., and Bonabeau, E. (2001). Self-Organization in Biological Systems. Princeton University Press.

Dale, M., Miller, J. F., Stepney, S., and Trefzer, M. A. (2019). A substrate-independent framework to characterize reservoir computers. Proceedings of the Royal Society A, 475(2226):20180723.

Jaeger, H. (2001). The "echo state" approach to analysing and training recurrent neural networks-with an erratum note. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report, 148(34):13.

Lukoševičius, M. and Jaeger, H. (2009). Reservoir computing approaches to recurrent neural network training. Computer Science Review, 3(3): 127-149.

Lymburn, T., Algar, S., Small, M., and Jüngling, T. (2021). Reservoir computing with swarms. Chaos, 31(3).

Reynolds, C. W. (1987). Flocks, herds and schools: A distributed behavioral model. In Proceedings of the 14th annual conference on Computer graphics and interactive techniques, pages 25-34.

Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., and Hirose, A. (2019). Recent advances in physical reservoir computing: A review. Neural Networks, 115:100-123.

Uehara, K., Inoue, K., Kuniyoshi, Y., and Nakajima, K. (2024). Ouroborotic swarm control: Collective dynamics as a reservoir. In ALIFE 2024: Proceedings of the 2024 Artificial Life Conference, Artificial Life Conference Proceedings, page 63.

Wringe, C., Trefzer, M., and Stepney, S. (2025). Reservoir computing benchmarks: a tutorial review and critique. International Journal of Parallel, Emergent and Distributed Systems, pages 1-39.

# Motion as the Source of the Spirits: Recording, Preservation, and Generation 

Aina Kojima ${ }^{1}$, Yuki Tanaka ${ }^{1}$, Kosuke Shikata ${ }^{1}$, Eiko Matsuda ${ }^{1}$, and Seiichiro Katsura ${ }^{1}$<br>¹Department of System Design Engineering, Keio University, Japan<br>kojima@katsura.sd.keio.ac.jp


#### Abstract

We argue that the "spirits," a distinct sense of vitality, resides in motion. Our research introduces a "motion-copying system" that achieves the precise extraction and reproduction of motion by recording both position and force trajectories with high fidelity. When we recorded and replayed the refined motions of a skilled calligrapher, the system exhibited an overwhelming sense of "vitality," as if the calligrapher's "spirits had been transferred into the machine". Currently, our work focuses on the preservation and analysis of this motion's temporal dynamics. Future work will advance from the passive preservation of "spirits" toward its active and dynamic generation. By integrating imitation learning with the logical reasoning of large language models (LLMs), our ultimate goal is not the mere mimicry of motions, but to learn a holistic action policy that reproduces the deep, underlying characteristics of human movement, enabling novel and contextually appropriate behaviors.


## Submission type: Late Breaking Abstracts

## Introduction

When confronted with life, one inevitably senses a quality that may be called its "spirits"-a distinct sense of vitality. But what is the source of this "spirits" of life? Autonomy, homeostasis and evolvability are among the many factors that define the nature of life. Yet these are only necessary conditions and cannot serve as sufficient ones. Evolutionary agents in simulations, oil droplets that maintain their structure, and autonomous robots each embody fragments of life-like qualities. However, none of them exhibit the overwhelming vitality-the "spirits"-that compels universal recognition as life.

Through this work, we argue that the "spirits" resides in motion. Within the field of artificial life, the idea that "motion" constitutes life has been discussed repeatedly. Rodney Brooks demonstrated that life-like robust behavior can emerge from simple interactions between a body and its environment through his robot design principle known as the "subsumption architecture" Brooks (2001). Morimoto and Ikegami, using agent-based simulations, showed that as life

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-172.jpg?height=483&width=844&top_left_y=826&top_left_x=1073)
Figure 1: Calligraphy reproduction of the character "Kokoro" (spirit) using the motion-copying system.

dynamically explores its environment, it acquires sensorimotor coordination grounded in its own embodiment, which in turn leads to the life-specific process of "meaning generation" Noë (2004); Morimoto and Ikegami (2004). While these examples highlight the decisive role of motion in lifelike intelligence, we extend this line of thought to assert that it is in motion itself that the "spirits" of life emerges.

## Motion-copying System

We have realized a technology which is called "motioncopying system" for the first time in the world, achieving the precise extraction of motion alone from living beings Yokokura et al. (2009). Motion is generated through the dual foundation of position and force. However, within the context of control theory, it has long been considered impossible to achieve high-precision control of both position and force simultaneously in robotics. Efforts to maintain positional accuracy typically render motion stiff and unnatural, while prioritizing softness and natural force modulation leads to oscillations and system instability. Despite such technical challenges, we succeeded in developing a motioncopying system capable of recording and replaying both position and force trajectories with high fidelity, by employing a bilateral control architecture that enables the simultaneous

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-173.jpg?height=362&width=841&top_left_y=267&top_left_x=155)
Figure 2: Timeline of our project for the recording, preservation, and generation of "spirits."

regulation of position and force. Unlike conventional motion capture, which acquires only kinematic data (positional information), our system also records interaction forces (force information) inherent in motion. This dual recording makes it possible to realize both precise positional control and natural force modulation at once. As a result, life-specific qualities of motion-position, flexibility, timing, and manner of touch-can be extracted and faithfully reproduced.

## Previous Work: Recording Calligraphy Motion and Emergence of Spirits

Fig. 1 shows the appearance of the calligraphy robot equipped with the motion-copying system. The natural motions of a human using a brush-encompassing both position and force-can be recorded with extreme precision and replayed as brush strokes. By having a skilled calligrapher hold the brush and write naturally, we recorded and reproduced the calligrapher's refined motions (position and force). Beyond the mere success of motion-copying, what was truly astonishing was the overwhelming sense of "vitality" embodied in the precise brush motions-as if an invisible calligrapher were holding and guiding the brush. The calligrapher himself remarked that it felt "as though my spirits had been transferred into the machine," while the experimenters also experienced the uncanny sensation that the calligrapher had possessed the brush robot. This finding, within the lineage of artificial life research that asserts motion creates life, fundamentally questions the very essence of lifethe "spirits" residing in precise motion itself-and the lifelike quality of motion and physicality.

## Current Work: Preservation and Analysis of Temporal Dynamics

In the presentation, we will further discuss which aspects of motion evoke this "spirits" through a time-series analysis of the positional data from each joint of the calligraphy robot. Rather than treating motion simply as a sequence of displacements, our approach views temporal structurethe rhythm, acceleration profiles, and coordination across joints-as potential carriers of expressive qualities. By ana-
lyzing these features over time, we aim to uncover the latent dynamics that allow certain motions to be perceived as gentle, powerful, or even imbued with vitality. Such an analysis not only helps us to characterize the structural signatures of "living" motion but also opens a pathway toward generative models that can synthesize new behaviors based on these principles. Ultimately, we envision this line of research contributing to the creation of robotic motions that transcend mechanical imitation, enabling machines to convey a sense of presence and spirit in their interactions with humans. The timeline of our project is shown in Fig. 2. In this way, the study of time-series patterns in motion becomes a bridge between quantitative analysis and the qualitative experience of "aliveness," offering insights into how artificial systems might one day embody lifelike expressivity.

The motion-copying system is expected to serve as a means of preserving and transmitting embodied knowledge. It is a novel recording system that is pre-linguistic and prerepresentational certainly, but at the same time, it also suggests the possibility of functioning as a system for recording and preserving the "spirits." As in Descartes' mind-body dualism, the "spirits" may indeed be separable from the body, existing as an immortal entity.

## Future Work: AI and Generation of Spirits

Our research marks a significant conceptual shift, advancing from the passive preservation of the "spirit" within motion toward its active and dynamic generation. We integrate two key capabilities to achieve this. First, imitation learning learns a human action policy by capturing nuanced statistical regularities, enabling the generation of adaptive motions with a repertoire far richer than simple data playback. Second, we utilize the logical reasoning inherent in large language models (LLMs) to facilitate long-term task planning and the sequencing of complex actions. The ultimate goal of this synthesis is not the mere mimicry of original motions, but to learn a holistic action policy that reproduces the deep, underlying characteristics that make a motion appear authentically human, thereby enabling novel, sociallyaware, and contextually appropriate behaviors.

## References

Brooks, R. (2001). The relationship between matter and life. Nature, 409(1):409-411.

Morimoto, G. and Ikegami, T. (2004). Evolution of plastic sensorymotor coupling and dynamic categorization. In Artificial Life IX: Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems. The MIT Press.

Noë, A. (2004). Action in Perception. Cambridge, MA: MIT Press.
Yokokura, Y., Katsura, S., and Ohishi, K. (2009). Stability analysis and experimental validation of a motion-copying system. IEEE Transactions on Industrial Electronics, 56(10):39063913.

# Society of Mind on Humanoid ALTER3 with Concurrent Modular Agents 

Takahide Yoshida ${ }^{1}$, Norihiro Maruyama ${ }^{1,2}$, Atsushi Masumoti ${ }^{1,2}$, and Takashi Ikegami ${ }^{1,2}$<br>${ }^{1}$ The University of Tokyo, Komaba, Japan<br>${ }^{2}$ Alternative Machine Inc.<br>\{yoshida, maruyama, masumori, ikeg\}@sacral.c.u-tokyo.ac.jp


#### Abstract

We present a concurrent modular agent (CMA) architecture that operationalizes Minsky's Society of Mind on a humanoid robot (ALTER3) deployed for months in a public venue. Multiple LLM-based modules run asynchronously, exchanging messages via a shared vector pool. We show self-organized k-lines-default memory-organizing assemblies and paired inner-dialogue modules-and report improved conversational Autonomy, Valence, and Physicality when all modules are active versus a dialogue-only ablation. To our knowledge, this is the first long-running public embodiment of asynchronous LLM module clusters on a humanoid platform, evidencing emergent self-regulation and identity formation. We detail the architecture, activation statistics, and evaluation protocol, and discuss implications for embodied cognitive architectures.


## Submission type: Late Breaking

Data/Code available at: https://osf.io/u8vfk/

## Introduction

Natural biological systems exhibit asynchronous and embodied intelligence, which has been increasingly explored in artificial systems (Pfeifer and Bongard, 2006; Brooks, 1991; Corkill, 1991). As proposed in Minsky's "Society of Mind" (Minsky, 1986), intelligence emerges from interactions of specialized agents rather than unified processing. While traditional approaches like Brooks' subsumption architecture (Brooks, 1986) have explored this concept using phisical robot, recent asynchronous agents (AL et al., 2024; González et al., 2024) remain limited to simulation environments. We propose a novel asynchronous distributed architecture using multiple interacting LLM modules, implemented on the humanoid robot ALTER3(Yoshida et al., 2023, 2024; Masumori et al., 2023) for daily operation in public spaces over several months (Fig.1(A)). This represents the first practical application of a real robot driven by asynchronous parallel modules-realizing the "society of mind" with precise autonomous behaviors emerging from module interactions.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-174.jpg?height=443&width=839&top_left_y=833&top_left_x=1077)
Figure 1: Overview of the CMA framework. (A) Humanoid robot ALTER3 with 43 pneumatic DOFs. (B) Multiple functional modules run concurrently, exchange messages, and share high-dimensional state vectors via shared stores.

## Concurrent Modular Agent Framework

We propose a modular agent framework, the Concurrent Modular Agent (CMA), in which multiple LLM-based modules operate asynchronously and interact via naturallanguage messages (Fig. 1B)(Maruyama et al., 2025). The framework comprises three core components: (i) a set of functional modules, (ii) a shared vector-based memory store, and (iii) an inter-module communication mechanism. Each module executes a specific subtask (perception, memory operations, planning, motivation, action, etc.). Module outputs are embedded and written to the shared store for later retrieval. in addition, modules exchange short messages for direct coordination. This yields a scalable, fault-tolerant system in which concurrent processes influence behavior through a common latent substrate.

Applied to ALTER3, CMA adopts a three-layer architecture (Fig. 2). The Hardware layer handles motor control and multi-modal I/O (camera, microphone). The Base layer comprises 11 LLM-based modules, including a recentmemory summarizer, a Desire module proposing actions from visual context, three Inner Dialogue modules (Magi $A / B / C$ ) providing complementary priors, and a selective memory cleaner preventing overflow. The Meta layer

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-175.jpg?height=422&width=721&top_left_y=287&top_left_x=214)
Figure 2: CMA architecture on ALTER3. Twenty modules are organized into three layers: hardware, base, and meta. The hardware processes camera/microphone I/O and motor control; base modules implement core LLM functions; summarize, inner dialogue, image description modules

hosts monitors assessing global state and generating autobiographical traces. Some modules determine their activation based on these meta-reports. This prevents the system from getting stuck in particular states. Together, the layers realize a Society-of-Mind control stack. With this system, ALTER3 has been continuously engaging in dialogue with humans in a public space (Venice Biennale Architecture 2025) over several months.

## Synchronous Activity of Modules

Minsky refers to a group of simultaneously active agents as a "k-line." This is a frame of mind that self-organizes to adapt to a certain state. We analyzed ALTER3's 11 base modules by counting simultaneously firing pairs (see Figure 3A,B). We found that memory-organizing modules (e.g., summarizer and cleaner) co-activate with other modules frequently and remain persistently active, effectively forming a defaultmode K-line. Magi $A$ and Magi $B$ tend to operate as a dialogic pair, and, on rare occasions, all modules become active at once.

## Semantic Analysis of ALTER3's Conversation

We evaluated the conversational data using GPT across three dimensions: Autonomy, Valence, and Physicality(see Figure 3C). We conducted a comparative analysis between two conditions: when all non-dialogue modules were deactivated versus when they were activated. ALTER3's conversational outputs are influenced by the integrated outputs from all modules, as well as self-descriptions generated from memory by the autobiographical memory module. Our analysis revealed that performance scores tended to be higher when all modules were activated. This effect was particularly shown for Valence, where ALTER3 demonstrated increased positive expressions. This is because ALTER3's personality was created through autobiographical memory.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-175.jpg?height=817&width=794&top_left_y=296&top_left_x=1105)
Figure 3: Analysis of ALTER3's activity data. (A) Example of daily activity records of base layer modules. Base modules exhibit heterogeneous duty cycles rather than uniform periodicity. Memory operators (Summarize, MemoryClear) show sustained background activity, whereas Magi and Desire fire in shorter, context-triggered bouts. (B-1) Heatmap of simultaneously active module pairs (average of 6 days of data). (B-2) Top 10 simultaneously active module pairs(average of 6 days of data). (C) Results of evaluating ALTER3's conversation data with humans using GPT (average of 6 days of data). When all non-dialogue modules are enabled, conversational quality improves versus a dialogueonly ablation, especially for Valence, consistent with contributions from autobiographical memory and meta-level monitoring.

## Discussion

Our modular architecture shows how language-based AI components can exhibit life-like properties when embedded in a physical robot, including self-regulation, identity formation, and context-sensitive decision-making. ALTER3 shows that combining LLMs, asynchronous processing, and physical feedback can create complex behaviors from simple interactions. We found that k -lines self-organize within our system and contribute to memory and personality structuring. Unlike past symbolic approaches, our system shows how such architectures can be realized using modern LLMs in physically embodied agents. This opens possibilities for grounding cognitive architectures in real-world interactions while maintaining the compositional richness envisioned in the Society of Mind.

## Acknowledgements

This work was supported in part by the Swiss National Science Foundation (SNSF), grant no. 10.002.211 and the NEXT Grant-in-Aid for Scientific Research "Exploring the Qualia Structure with Large Language Models and a Humanoid Robot" (Grant No. 24H01546)

## References

AL, A., Ahn, A., Becker, N., Carroll, S., Christie, N., Cortes, M., Demirci, A., Du, M., Li, F., Luo, S., Wang, P. Y., Willows, M., Yang, F., and Yang, G. R. (2024). Project sid: Manyagent simulations toward ai civilization.

Brooks, R. A. (1986). A robust layered control system for a mobile robot. In IEEE Journal of Robotics and Automation, volume 2, pages 14-23. IEEE.

Brooks, R. A. (1991). Intelligence without representation. Artificial intelligence, 47(1-3):139-159.

Corkill, D. D. (1991). Blackboard systems. AI Expert, 6(9):40-47.
González, J. L., Cheng, L., Shen, S., Liò, P., and Bian, Y. (2024). Lyfe agents: Brain-inspired low-cost generative agents for social interactions. arXiv preprint arXiv:2401.10061.

Maruyama, N., Yoshida, T., Sato, H., Masumori, A., Johnsmith, and Ikegami, T. (2025). A concurrent modular agent: Framework for autonomous llm agents.

Masumori, A., Maruyama, N., and Ikegami, T. (2023). Personogenesis through imitating human behavior in a humanoid robot "alter3". In Proceedings of the 2023 Artificial Life Conference (ALIFE 2023), pages 521-529. MIT Press.

Minsky, M. (1986). The Society of Mind. Simon and Schuster.
Pfeifer, R. and Bongard, J. (2006). How the body shapes the way we think: a new view of intelligence. MIT press.

Yoshida, T., Baba, S., Masumori, A., and Ikegami, T. (2024). Minimal self in humanoid robot "alter3" driven by large language model. volume ALIFE 2024: Proceedings of the 2024 Artificial Life Conference of Artificial Life Conference Proceedings, page 53.

Yoshida, T., Masumori, A., and Ikegami, T. (2023). From text to motion: Grounding gpt-4 in a humanoid robot "alter3".

# H-Lenia: A Hierarchical Extension of Lenia to Analyze Multi-level Emergence through Inter-level Feedback 

Tatsuki Furukawa ${ }^{1,2}$, Takaya Arita ${ }^{1}$ and Reiji Suzuki ${ }^{1}$<br>${ }^{1}$ Nagoya University, Japan<br>${ }^{2}$ furukawa.tatsuki.d1 @ s.mail.nagoya-u.ac.jp


#### Abstract

This paper presents H-Lenia, a hierarchical extension of Lenia designed for multi-level interactions. The experiment discovered a new organism exhibiting inter-layer feedback, with subsequent analysis revealing its hierarchical structure and dynamics.


## Submission type: Late Breaking Abstracts

## Introduction

Living systems exhibit hierarchical organization spanning from molecular to ecosystem levels, with emergent phenomena arising through cross-level interactions. This represents an inclusive hierarchy where each level operates at distinct spatial and temporal scales with specific interaction rules. For example, in biological organisms, the behavior of cells and organs (micro-level) influences individual organism dynamics (macro-level), while conversely, the overall individual organism state regulates the activity of those components.

Lenia [Chan, 2019, 2020] extends Conway's classical two-dimensional cellular automaton, the Game of Life by introducing continuous time, space, and states, along with generalized transition rules. This framework allows for more flexible configurations and has demonstrated the emergence of autonomous patterns reminiscent of microorgan-isms-a phenomenon impossible with previous discrete models. Lenia established an entirely new model system and research direction for ALIFE studies exploring universal principles of life. Since its proposal, Lenia has been extended in various directions such as asymptotic rule modifications (A-Lenia) [Kawaguchi et al., 2021], mass conservation (Flow-Lenia) [Plantec et al., 2025], particle dynamics (Particle-Lenia) [Mordvintsev et al., 2022], and resourceenvironment interactions (Lenia in a Petri Dish) [Suzuki et al., 2023]. However, while the rich parameter space of Lenia enables diverse self-organizing patterns, its singlelayer architecture limits investigation of the cross-scale interactions that characterize biological hierarchies described above.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-178.jpg?height=433&width=673&top_left_y=831&top_left_x=1162)
Figure 1: Schematic overview of H-Lenia.

This paper proposes H-Lenia (fig. 1), a hierarchical extension of Lenia designed to investigate multi-level emergence through inter-level feedback, and reports morphological and informational analyses of emergent organisms.

## H-Lenia: A Hierarchical extension of Lenia

H-Lenia consists of a hierarchical structure of $n$ layers, each implementing a multi-channel, multi-kernel Lenia world, where upper layers have larger spatial and slower temporal scales than their lower-level counterparts, as illustrated in fig. 1. Each layer $i$ follows the governing equation:

$$
\begin{aligned}
& A_{i}^{t+\Delta t}= \\
& \left\{\begin{array}{rr}
{\left[\left[A_{i}^{t}+\Delta t G\left(K * A_{i}^{t}\right)\right]_{0}^{1}+\sum_{j \in N(i)} k_{j i} \cdot E_{j i}^{t}\left(\Delta_{j \rightarrow i}^{t}\right)\right]_{0}^{1}} \\
A_{i}^{t} & \text { if }\left(t \bmod T_{i}^{S} \cdot \Delta t\right)=0 \\
\text { otherwise }
\end{array}\right.
\end{aligned}
$$

where $T_{i}^{S}$ controls the update frequency of layer $i, N(i)$ denotes adjacent layers, $k_{j i}$ represents the interaction strength from layer $j$ to layer $i, E_{j i}$ is either attraction ( $E_{A}$ ) or repulsion ( $E_{R}$ ) applied to the resolution-adjusted difference between layers $A_{j}$ and $A_{i}\left(\Delta_{j \rightarrow i}^{t}\right)$. The update process at each time step consists of three stages: 1) Update Activation, where the model determines whether each layer $i$ updates based on its temporal scale $T_{i}^{S}$; 2) Intra-Layer Dynamics, where state transitions are computed in parallel within each layer based on the original rule of Lenia (i.e., Aquarium);

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-179.jpg?height=410&width=670&top_left_y=269&top_left_x=237)
Figure 2: Snapshots of an emergent organism. Top row, from left to right: layers $L_{1}, L_{2}$, and $L_{3}$. Bottom row: a time sequence (before, present, after) of the multi-layer overlay. Yellow circles indicate the organism. Video link: https://figshare.com/s/8ee806be077fe1c47c84

3) Inter-Layer Interactions, where directional forces are applied to the state values of the focal layer $i$ from its adjacent layers $j$ s. Attraction ( $E_{A}$ ) pulls the focal layer $i$, while repulsion $\left(E_{R}\right)$ drives the focal layer $i$ away from its adjacent layers $j \mathrm{~s}$, with effects scaled by interaction strength $k_{j i}$ and modulated by boundary damping. States are mapped by upscaling (copying) or downscaling (averaging) to reconcile different resolutions.

## Pilot experiments and analyses

Experiments were conducted using a three-layer H-Lenia model. We denote the bottom, middle, and top layers as $\boldsymbol{L}_{1}$, $\boldsymbol{L}_{\mathbf{2}}$, and $\boldsymbol{L}_{\mathbf{3}}$, respectively. This setup establishes a micro-to-macro hierarchy: $L_{1}$ serves as the fast, high-resolution "micro-level" ( $1536 \times 1536, T_{1}^{S}=1$ ), while $L_{2}(384 \times 384$, $\left.T_{2}^{S}=4\right)$ and $L_{3}\left(96 \times 96, T_{3}^{S}=16\right)$ function as progressively slower, lower-resolution "macro-levels." The system was initialized with Aquarium patterns placed only on the $L_{1}$, with all other layers set to zero, anticipating the emergence of multi-level biological structures from the bottom layer.

By exploring combinations of interaction types between layers ( $E_{j i}$ : Attraction(A) or Repulsion(R)) and their strengths $\left(k_{j i}\right)$, we discovered the emergence of stable organism patterns based on interactions among three layers at the following parameter: $E_{23}\left(k_{23}\right) / E_{32}\left(k_{32}\right) E_{12}\left(k_{12}\right) / E_{21}\left(k_{21}\right)=\mathrm{R}(0.6) / \mathrm{A}(0.5) \mathrm{R}(0.9) / \mathrm{A}(0.4)$, meaning that in adjacent layers, repulsive forces were applied to the upper layer, while attractive forces were applied to the lower layer.

Figure 2 shows snapshots of the population of emergent organisms and their collective movement. The organisms maintained a size close to that of the $L_{3}$-scale Aquarium organism; however, they possessed an entirely different hierarchical structure, as schematically illustrated in fig. 3. This organism has a structure where roles are divided across different scale layers (fig. 3 (a)). Analysis revealed that the whole structure of organisms is maintained through the fol-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-179.jpg?height=405&width=670&top_left_y=269&top_left_x=1160)
Figure 3: Schematic of emergent organism structure. (a) Steady state, (b) During movement.

lowing mechanism: An increase in the state value of the $L_{1}$ from the initial state triggers an increase in the state value of the $L_{2}$, which in turn causes a further increase in the $L_{1}$, creating a positive feedback loop. This is based on a repulsion-attraction coupling where $L_{1}$ amplifies $L_{2}$ growth (repulsion effect), and $L_{2}$ concentrates $L_{1}$ values (attraction effect). $L_{1}$ maintains moderate values through intra-layer dynamics, forming a foundation that sustains the entire organism. Similarly, positive feedback between $L_{2}$ and $L_{3}$ further increases their respective state values. In the $L_{3}$, a hollowed-out, doughnut-shaped structure emerged, possibly due to the intra-layer dynamics and constraints based on the Aquarium rule. This becomes the organism's body. Consequently, $L_{2}$ develops a thin ring structure serving as the organism's skeleton with elevated state values resulting from combined influences from $L_{1}$ and $L_{3}$. It should be noted that the stability of the ring structure appears to determine the organism's behavior. When part of the $L_{2}$ skeleton collapsed into a U-shape, the organism moved in the opposite direction with the collapsed side facing backward (fig. 3 (b)). This collapse is thought to have occurred because chaotic influences caused the state value of the $L_{1}$ to exceed that of the $L_{2}$.

We observed the emergence of these organisms in all 10 trials under the same experimental conditions. The transfer entropy (TE) analyses of state values between adjacent layers showed that upward TE exceeded downward TE, with the absolute amount of information flow decreasing at higher layers. This implies that consistent bottom-up information flow combined with intermediate mutual interactions may contribute to stable yet mobile behavior for emergent organisms with hierarchical structures.

## Conclusion

We proposed H-Lenia and demonstrated emergent organisms with hierarchical organization, revealing new possibilities for investigating multi-level emergence of life.

## References

B. W.-C. Chan. Lenia - biology of artificial life. Complex Systems, 28:251-286, 2019. doi: 10.25088/complexsystems.28.3.251.
B. W.-C. Chan. Lenia and expanded universe. In The 2020 Conference on Artificial Life, volume 32, pages 221-229, 2020. doi: 10.1162/isal_a_00297.
T. Kawaguchi, R. Suzuki, T. Arita, and B. W.-C. Chan. Introducing asymptotics to the state-updating rule in Lenia. In Proceedings of the 2021 Conference on Artificial Life, page 91, 2021. URL https://doi.org/10.1162/isal_a_00425. Paper No: isal_a_00425, 91 (3 pages).
A. Mordvintsev, E. Niklasson, and E. Randazzo. Particle Lenia and the energy-based formulation. https://google-research.github.io/ self-organising-systems/particle-lenia/, 2022.
E. Plantec, G. Hamon, M. Etcheverry, P.-Y. Oudeyer, C. MoulinFrier, and B. W.-C. Chan. Flow-Lenia: Emergent evolutionary dynamics in mass conservative continuous cellular automata. Artificial Life, 31(2):228-248, 05 2025. ISSN 10645462. doi: 10.1162/artl_a_00471. URL https://doi. org/10.1162/artl_a_00471.
R. Suzuki, K. Asakura, and T. Arita. Lenia in a petri dish: Interactions between organisms and their environment in a Lenia with growth based on resource consumption. In Proceedings of the 2023 Conference on Artificial Life, pages 242-244, 2023. doi: 10.1162/isal_a_00613. URL https : //doi.org/10.1162/isal_a_00613.

# Modeling Emergent Patterns of Life in Dynamic Galactic Systems 

Harrison B. Smith ${ }^{1,2, \dagger}$ and Lana Sinapayen ${ }^{3,4}$,<br>${ }^{1}$ Earth-Life Science Institute, Tokyo Institute of Technology, Ookayama, Meguro-ku, Tokyo, Japan<br>${ }^{2}$ Blue Marble Space Institute of Science, Seattle, Washington, USA<br>${ }^{3}$ Sony Computer Science Laboratories, Kyoto, Japan<br>${ }^{4}$ National Institute for Basic Biology, Okazaki, Japan<br>† hbs@elsi.jp


#### Abstract

Previously (Smith and Sinapayen, 2024) we built an agentbased model of life (1) spreading throughout the galaxy and (2) altering the observable characteristics of planets. Those results indicated that these two processes lead to a detectable subset of planets with anomalously high positive correlations between their spatial locations and characteristics. This indicates that we can use ALIFE models to identify the existence of life amongst a group of planets, even without being able to detect life on any planet in isolation. Our biosignature is unique by not requiring any judgements on individual features of life or technology. This agnosticism means we are not restricted to Earth-like life, planets, habitability, or to simple anomaly detection. Here we investigate the effect of stellar motion under different conditions on these results, aiming to bring greater realism to our model.


## Submission type: Late Breaking Abstract

Model code available at: https://github.com/ hbsmith/TerraformingAgents

## Introduction

The search for a second instance of life is one of the greatest problems of modern science. Outside of creating an artificial origin of life on Earth (Centler et al., 2003; Helman, 2022), the primary targets for the search for life are planets inside or outside the solar system. Realistically, there are just a few locations to search for alien life within the solar sys-tem-Mars, Venus, and a handful of icy moons. Outside the solar system, opportunities are nearly unlimited. But there's a catch: It is difficult to attribute, with certainty, features of exoplanets to extraterrestrial life (Tasker et al., 2017). Simple spectral biosignatures are susceptible to false positives (Smith and Mathis, 2022); technosignatures reduce this susceptibility at the expense of strong assumptions about potential underlying life and its technologies.

We have developed an agnostic approach to exoplanet life detection that overcomes these limitations by using properties that emerge on the scale of groups of planets, without the need for a "smoking-gun" single-planet level biosignature. In previous modeling work (Smith and Sinapayen, 2024), we
showed that if life can i) spread between star systems, and ii) affect the observable properties of a planet, then a robust signature of life (with very few false positives) emerges, defined by correlations between planet characteristics and their locations. This work demonstrated the utility of using ALIFE models not only to directly try to create life in silico, but also as a tool to guide the search for life in the galaxy.

## Initial Modeling and Results

Our analysis methodology is built on previous work (Smith and Sinapayen, 2024). Briefly, we created an agent-based model to simulate life ${ }^{1}$ spreading between planets orbiting different host stars. The model is initialized with 1000 planets ( 1 of which is inhabited/capable of interstellar travel), fixed in a frame of reference, and uniformly randomly distributed in a continuous 3D volume. All planets are assumed to be in different star systems (i.e., we assume one planet per star). "Planets" have compositions representing the observable characteristics of the planet, and "life" agents have compositions representing the observable characteristics of the planet which the life originates from. Both "life" and "planets" compositions are abstractions represented by vectors of 10 real numbers (each element $\in[0,1]$ ), used as a means to compare the compatibility of any given life and planet. During simulation, inhabited planets ${ }^{2}$ create and send out a "life" agent towards a target uninhabited planet at a constant average velocity ${ }^{3}$. The target planets are selected

[^9]based on proximity in position and in composition. When life arrives at a planet, the planet's composition is modified (as it becomes inhabited) based on both the initial composition of the planet and the composition of the incoming life. Life originating from that planet will then reflect its new composition. We assume that the time required for planetary alteration is negligible compared to the time required to travel between planets. Planets altered by life create and send out "life" agents at a fixed rate, provided suitable target planets exist. The simulation ends when all planets are inhabited, or no suitable target planets remain.

We do not know how many planets in any region of space might contain life (i.e., we do not know which timestep in our simulation is closest to reality). To simulate life detection in the face of this uncertainty, we pretend that we do not know which planets have been modified in our simulations, and make it our task to identify them. We first use the Mantel test-a statistical test common in ecological science Mantel, 1967). This quantifies the correlation of (a) the spatial distance between all pairs of planets, and (b) the distance between the vectors representing their characteristics. Planets that are close in space and characteristically similar drive this correlation up (Figure 1). When planets are clustered by their characteristics, then filtered to clusters localized in space, we were able to pinpoint groups of planets most likely to have been modified by life with almost no false positives.

## On-going Model and Analysis Improvements

Our approach to life detection is not simply to look for similar planets, but to look for planets whose similarity is related to their position in space. The difference can be demonstrated by shuffling the composition of only the inhabited planets at each timestep, and quantifying the effect this has on the mantel coefficient (Figure 1, grey line). This shuffling lowers the mantel coefficient, while retaining low p-value, indicating that position plays an important roll in driving up the detected correlations in our simulation.

A drawback of our initial implementation that we aim to improve with this work is understanding the effects of stellar dynamics on our results. Stars move relative to each other in space. Even if life were to spread between nearby stars as we have proposed, on long timescales these stars will move apart, changing the composition of each stars stellar neighborhood. As a first pass to quantify this effect, we evaluated the relative motion of stars using a time-evolution of a simulated globular cluster (using the Plummer model (Aarseth et al., 1974)). We find that in 1 million years of stellar evolution 73\% of the distances between star pairs are less than $10 \%$ different than their absolute distances at $\mathrm{t}=0$ (Figure 2).

Globular clusters however are relatively tightly bound in space. In on-going work, we are analyzing the dynamics of stars in the solar neighborhood using the Fifth Catalogue of

[^10]![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-183.jpg?height=298&width=607&top_left_y=338&top_left_x=1187)
Figure 1: Mantel coefficient and p -value as a function of the ratio of planets terraformed, for the same data analyzed in (Smith and Sinapayen, 2024). However, instead of only looking at the mantel coefficient of the space, we also look at the mantel coefficient of the terraformed planets when shuffled in order to highlight that our methodology is not simply looking for similar planets, but is looking for planets whose similarity is correlated with their location in space. Lines show the mantel coefficient of all planets at each timestep (black) and when inhabited planet locations are shuffled (grey). Corresponding p-values are shown (red for non-shuffled, pink for shuffled).

Nearby Stars (Golovin et al., 2023). This is a curated dataset of properties (including positions and velocities) from all currently identified objects within 25 pc of the sun, including for 5230 stars. By using up-to-date astrophysical data, we hope to better constrain our model's efficacy of life detection under realistic conditions useful for future Earth-based observations.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-183.jpg?height=595&width=608&top_left_y=1594&top_left_x=1197)
Figure 2: Heatmap showing the absolute difference in the distance matrix of stars at $\mathrm{t}=0$ and $\mathrm{t}=1$ million years of stellar evolution.

## References

Aarseth, S. J., Henon, M., and Wielen, R. (1974). A Comparison of Numerical Methods for the Study of Star Cluster Dynamics. Astronomy and Astrophysics, 37:183-187.

Centler, F., Dittrich, P., Ku, L., Matsumaru, N., Pfaffmann, J., and Zauner, K.-P. (2003). Artificial Life as an Aid to Astrobiology: Testing Life Seeking Techniques. In Banzhaf, W., Ziegler, J., Christaller, T., Dittrich, P., and Kim, J. T., editors, Advances in Artificial Life, pages 31-40, Berlin, Heidelberg. Springer.

Golovin, A., Reffert, S., Just, A., Jordan, S., Vani, A., and Jahreiß, H. (2023). The Fifth Catalogue of Nearby Stars (CNS5). Astronomy \& Astrophysics, 670:A19.

Helman, D. S. (2022). Finding or Creating a Living Organism? Past and Future Thought Experiments in Astrobiology Applied to Artificial Intelligence. Acta Biotheoretica, 70(2):13.

Mantel, N. (1967). The detection of disease clustering and a generalized regression approach. Cancer research, 27(2_Part_1):209-220.

Smith, H. B. and Mathis, C. (2022). The Futility of Exoplanet Biosignatures. arXiv:2205.07921 [astro-ph, q-bio].

Smith, H. B. and Sinapayen, L. (2024). An Agnostic Biosignature Based on Modeling Panspermia and Terraformation.

Tasker, E., Tan, J., Heng, K., Kane, S., Spiegel, D., and the ELSI Origins Network Planetary Diversity Workshop (2017). The language of exoplanet ranking metrics needs to change. Nature Astronomy, 1(2):0042.

# Towards Physicochemical Foundations of Chemical Learning 

Marco Tuccio ${ }^{1,2}$, Jason W. Rocks ${ }^{1}$, and Joshua E. Goldford ${ }^{1}$<br>${ }^{1}$ Dayhoff Labs, Inc. Cambridge, MA, USA<br>${ }^{2}$ Universitat Pompeu Fabra, Barcelona, Spain<br>marco.tuccio95@gmail.com

## Submission type: Late Breaking Abstracts


#### Abstract

A defining feature of the transition from chemistry to biology has been the development of increasingly sophisticated information processing and learning capabilities. However, the physicochemical mechanisms and constraints that first enable chemical learning remain enigmatic. We investigate how thermodynamic and kinetic parameters enable chemical reaction networks to perform learning tasks, focusing on linear regression as a minimal computational primitive. Through analytical and optimization methods, we identify parameter regimes where steady-state responses exhibit linear behavior, both in simplified linear models and in more complex, physically realistic nonlinear systems.


## Introduction

How can chemical systems acquire the capacity to learn? Recent perspectives suggest that emergent physicochemical properties endowed early chemical systems with algorithmic capabilities, driving the evolution toward life with increasingly complex information processing abilities Walker and Davies (2013). Yet, current work in molecular programming explores learning and computation through highly abstract frameworks such as imposing Boolean logic inspired by silicon computing Cardelli et al. (2018), focusing on topological formalisms Vasić et al. (2022), or neglecting thermodynamic and kinetic constraints that govern chemical systems Mordvintsev et al. (2023). This leaves a gap in understanding how catalysis, autocatalysis, free-energy driving forces, and redundancy might ground learning in actual chemistry.

To address these questions, we analyze how steady-state concentrations in chemical reaction networks (CRNs) depend on physicochemical features such as thermodynamic, kinetic, and topological parameters in simple toy models. Building on these insights, we demonstrate that larger, chemically motivated CRNs can be tuned to perform basic machine learning tasks, establishing a path toward scaling CRN models to capture more complex behaviors under increasingly realistic biochemical constraints.

Our investigation starts from the simplest possible task: linear regression. The inflow rate of certain species in the

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-186.jpg?height=716&width=794&top_left_y=849&top_left_x=1096)
Figure 1: we tune a linear three-reaction cycle to any positiveslope linear response and map the parameter-space (see code)

CRN is interpreted as the input signal.

## Linear response in unimolecular networks

We start by analyzing linear response in an introductory toy example on linear CRNs (i.e., exclusively unimolecular). Dynamics are governed by:

$$
\frac{\mathrm{d} \overrightarrow{\mathbf{x}}}{\mathrm{~d} t}=\mathbf{K} \overrightarrow{\mathbf{x}}+\overrightarrow{\mathbf{k}}_{i n}-\delta \overrightarrow{\mathbf{x}}
$$

where $\overrightarrow{\mathrm{x}}$ is the concentration vector, $\mathbf{K}$ is the rate matrix containing first-order reaction rates, $\overrightarrow{\mathbf{k}}_{\text {in }}$ is the inflow vector, and $\delta$ is the dilution rate. At steady state:

$$
\hat{\mathbf{x}}=[\delta \mathbf{I}-\mathbf{K}]^{-1} \overrightarrow{\mathbf{k}}_{i n}
$$

Showing that steady-states depend linearly on inflow $\overrightarrow{\mathbf{k}}_{i n}$.
As an example, we get a three-species cycle (Fig. 1) with inflow to species $b$ to respond linearly on the steady state of species $a$ with any $w \in \mathbb{R}^{+}$slope (see code):

$$
\hat{a}=w \cdot k_{i n}
$$

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-187.jpg?height=752&width=1669&top_left_y=242&top_left_x=212)
Figure 2: a) randomly initialized $a_{4}$ string chemistry with super- and sub-linear response to inflow. b) optimized $a_{4}$ : inflow species $a$, output steady state âa $a$. c) inflow species aaaa, output steady state of âa $a$. Response optimized against $y=3 x$

where $w$ is the corresponding element of $[\delta \mathbf{I}-\mathbf{K}]^{-1}$ and depends on all kinetic parameters and the dilution rate.

We conclude that any linear chemostatted CRN can implement exact linear response, with slope in $\mathbb{R}^{+}$underdetermined with respect to kinetic parameters, providing flexibility in implementing desired linear transformations without detailed engineering of specific stoichiometry.

## Linear regression in nonlinear polymerization

While linear response of steady states to inflow rates is easily obtained in the linear regime, it is more difficult to achieve in real-world CRNs as they are are largely nonlinear. As a model for more physically realistic CRN topologies, we study an artificial string chemistry Moyer et al. (2021) composed of simple polymerization and breakdown bimolecular reactions, as exemplified by the $a 4$ topology composed of $a$ monomers and with maximum polymer length 4 in Fig.2.

Here we cannot count on linear algebra to find solutions, due to analytical intractability and the nonlinear nature of the ODE system, so we proceed via computational optimization of parameter values to search for solutions. To obtain the kinetics-dependent steady state response $\hat{\mathbf{x}}$, we minimize:

$$
\begin{aligned}
\hat{\mathbf{x}} & =\min _{\overrightarrow{\mathbf{x}}}\left\|\frac{\mathrm{d} \overrightarrow{\mathbf{x}}}{\mathrm{~d} t}\right\|^{2} \\
\frac{\mathrm{~d} \overrightarrow{\mathbf{x}}}{\mathrm{~d} t} & =\overrightarrow{\mathbf{f}}_{\mathbf{K}}(\overrightarrow{\mathbf{x}})
\end{aligned}
$$

Where $\mathbf{K}$ represents all system kinetic parameters and dilution. To search for a set of parameters $\mathbf{K}^{*}$ for which response is linear, we minimize $\mathbf{K}$ against a loss function that represents linearity of steady state response on inflow rates.

Convergence results. We train an a4 topology on linearity, which is achieved by very specific parameter sets
that correspond to sweet spots at the boundary between the more common sublinear and superlinear response of such networks (Fig.2a). Thermodynamical and kinetic qualitative patterns emerge: when setting the inflow input species to a monomer and the output species to a polymer (Fig.2b), the polymerization reaction $a+\mathrm{n} a \leftrightharpoons(\mathrm{n}+1) a$ becomes kinetically fast but thermodynamically hindered, having equilibrium strongly to the side of substrates. Meanwhile, the reaction in which the inflow monomer and the output polymer are both substrates in the formation of a longer polymer $a+(\mathrm{n}+1) a \leftrightharpoons(\mathrm{n}+2) a$, is kinetically slower but thermodynamically favored. In the inverse setup, when inflow (input) is set to a polymer and output to a shorter polymer or monomer, the pattern is flipped (Fig.2c). We see this pattern emerge also in bigger networks (not shown).

## Conclusions

Our trajectory is to scale our experiments on three axes: Physical realism: we aim to model increasingly realistic systems and neatly map simulation results to reality. Network complexity: add features such as autocatalysis, increased degeneracy, catalysis and free-energy driving. Hardness of tasks: progressively increase the difficulty of tasks to train CRNs on.

In work in progress, we already uncovered principles for sigmoidal response, analyzing how simple physical mechanisms may map to perceptron behavior. Our goal is to find crucial features that enable increasingly complex response behavior of chemical systems and find scaling laws for chemical computing and advancing the path to cellular supremacy Grozinger et al. (2019).

## Funding and acknowledgements

This work has been fully funded by Dayhoff Labs Inc. No conflict of interest to be declared. We thank Artemy Kolchinsky and Ricard Solé for precious discussions.

## References

Cardelli, L., Kwiatkowska, M., and Whitby, M. (2018). Chemical reaction network designs for asynchronous logic circuits. Nat. Comput., 17(1):109-130.

Grozinger, L., Amos, M., Gorochowski, T. E., Carbonell, P., Oyarzún, D. A., Stoof, R., Fellermann, H., Zuliani, P., Tas, H., and Goñi-Moreno, A. (2019). Pathways to cellular supremacy in biocomputing. Nat. Commun., 10(1):5250.

Mordvintsev, A., Randazzo, E., and Niklasson, E. (2023). Differentiable programming of chemical reaction networks. arXiv [q-bio.MN].

Moyer, D., Pacheco, A. R., Bernstein, D. B., and Segrè, D. (2021). Stoichiometric modeling of artificial string chemistries reveals constraints on metabolic network structure. J. Mol. Evol., 89(7):472-483.

Vasić, M., Chalk, C., Luchsinger, A., Khurshid, S., and Soloveichik, D. (2022). Programming and training rate-independent chemical reaction networks. Proc. Natl. Acad. Sci. U. S. A., 119(24):e2111552119.

Walker, S. I. and Davies, P. C. W. (2013). The algorithmic origins of life. J. R. Soc. Interface, 10(79):20120869.

# Of Flesh and Code: Alife-meets-Blife Project 

Juan M. Nadales ${ }^{1}$, Hiroki Kojima ${ }^{2}$, and Liam M. Longo ${ }^{1,3}$<br>${ }^{1}$ Earth-Life Science Institute, Institute for Future Science, Institute of Science Tokyo, Tokyo, Japan, llongo@elsi.jp<br>${ }^{2}$ Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan<br>${ }^{3}$ Blue Marble Space Institute of Science, Seattle, Washington, USA


#### Abstract

The Alife-meets-Blife project investigates how adaptive behaviors and emergent properties arise when biological organisms (Blife) and artificial life forms (Alife) interact within a shared ecosystem. The objective is to advance understanding of life's complexity beyond the genome by focusing on the dynamics of community formation and the conditions under which agency emerges and transforms. By integrating living and computational entities, we establish a novel experimental paradigm for exploring cooperation, competition, and resilience across natural and synthetic domains. This approach provides a framework for testing hypotheses about the transferability of agency and the mechanisms that enable mutualistic interactions between biological and artificial systems. The project is designed to engage the broader research community in rethinking the relationship between structure and function, genotype and phenotype, and biology and artificiality. By charting this new territory, Alife-meets-Blife aims to contribute to a deeper understanding of the principles that govern complex living systems, opening new directions for both theoretical and experimental research.


Submission type: Late-Breaking Abstract

## Introduction

Living systems are characterized by their capacity to adapt, self-organize, and generate emergent properties that extend beyond the sum of individual components. These properties are especially evident when organisms form communities, where interactions such as cooperation, competition, and exchange of information shape collective behavior and influence evolutionary trajectories. From an ecological perspective, community formation plays a vital role in maintaining stability and resilience through mechanisms such as resource partitioning, niche specialization, and mutualistic relationships Alimov (2010); Morin (2009). Such dynamics illustrate the complexity of life, which cannot be fully explained by genetic information alone but arises from multi-level interactions between organisms and their environments.

Artificial life has long provided a means to explore these principles through computational models and simulations

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-190.jpg?height=807&width=842&top_left_y=831&top_left_x=1073)
Figure 1: Alife-Blife Platform: A) Bioreator for growing cells. B) Computer where Alife simulation runs.

Macal (2020). Agent-based approaches, evolutionary algorithms, and adaptive regulatory networks have proven valuable for studying cooperation, resilience, and strategy evolution under controlled conditions. However, digital systems remain limited in their ability to capture the richness of biological organization, often lacking the metabolic, physical, and ecological dimensions that characterize natural systems. They also tend to oversimplify emergent behaviors, which in natural contexts depend on the non-linear interplay of genetic, epigenetic, and environmental factors Tagkopoulos et al. (2008). This gap highlights the need for integrative approaches that bring together biological and artificial domains.

Recent advances in synthetic biology have further demonstrated the potential for engineered systems to perform distributed computation, self-regulation, and community-level coordination Tamsir et al. (2011); Song et al. (2009); Macía
et al. (2012). These works show that even simple genetic circuits can generate collective behaviors, reinforcing the idea that computation and life are deeply intertwined processes. At the same time, conceptual efforts in philosophy of biology and cognitive science continue to refine the notion of agency as a key feature of living and life-like systems Barandiaran et al. (2009); Pickering (2024). This convergence of synthetic, computational, and theoretical approaches has motivated new frameworks that seek to bridge natural and artificial systems in order to better understand life's organizing principles Baltieri et al. (2023).

The Alife-meets-Blife project responds to this challenge by proposing a hybrid framework in which biological organisms and computational entities interact within a shared environment. This perspective opens new possibilities for examining the conditions under which adaptive behaviors emerge, agency is transformed, and communities achieve novel forms of organization, while also setting clear objectives: 1) Provide a methodological foundation through the implementation of the Alife-Blife platform (see Figure 1), enabling systematic exploration of emergent properties in hybrid systems; 2) Investigate the dynamics of hybrid communities and how interactions shape cooperation, competition, and resilience; 3) Examine the emergence and transformation of agency when living and artificial systems engage within shared environments Barandiaran et al. (2009); Pickering (2024); 4) Identify principles of life beyond the genome, emphasizing relational and ecological interactions that drive adaptability and collective organization. By bridging biological complexity with computational innovation, the project contributes to the growing effort to understand life, situating inquiry at the interface of biology, technology, and theory.

## Alife-Blife Platform

The Alife-Blife platform consists of two main components: 1) A simulation model of bacterial population growth based on the EVE framework developed in Tagkopoulos et al. (2008), and 2) A bioreactor system where E. coli cultures are maintained under controlled conditions. The EVE simulator provides a powerful environment for modeling adaptive regulatory networks, community-level dynamics, and predictive behaviors in microbial populations, making it especially suitable for capturing the nonlinear and emergent properties of living systems. The bioreactor, designed and constructed from scratch, incorporates precise regulation of agitation, temperature, nutrient concentration, and programmable pumping and extraction of additives and waste, creating a dynamic environment for E. coli populations. The coupling between the artificial (Alife) and biological (Blife) communities is achieved through a closed loop of information exchange: measurements of bacterial growth and concentration from the bioreactor are transmitted by a microcontroller (Arduino-based control system) to the simu-
lator, while outputs from the artificial agents are sent back as control commands that adjust bioreactor parameters such as pumping rates or nutrient composition. This messagepassing interface ensures that both domains evolve in mutual interaction, establishing a hybrid ecosystem where computational and biological populations co-adapt and co-regulate. Future extensions of the platform will integrate a flow cytometer into this communication loop, allowing direct and high-resolution quantification of bacterial populations, thus enhancing the fidelity and responsiveness of the coupled system.

## Acknowledgements

The work on this project has been funded by the John Templeton Foundation. The authors would like to acknowledge the collaboration of the Manufacturing and Design Division from Institute of Science Tokyo at Ookayama Campus for the manufacturing of some mechanical pieces for the bioreactor.

## References

Alimov, A. (2010). Biological diversity and the community structure of organisms. Inland Water Biology, 3:207-213.

Baltieri, M., Iizuka, H., Witkowski, O., Sinapayen, L., and Suzuki, K. (2023). Hybrid life: Integrating biological, artificial, and cognitive systems. Wiley Interdisciplinary Reviews: Cognitive Science, 14(6):e1662.

Barandiaran, X. E., Di Paolo, E., and Rohde, M. (2009). Defining agency: Individuality, normativity, asymmetry, and spatiotemporality in action. Adaptive behavior, 17(5):367-386.

Macal, C. M. (2020). Agent-based modeling and artificial life. Complex Social and Behavioral Systems: Game Theory and Agent-Based Models, pages 725-745.

Macía, J., Posas, F., and Solé, R. V. (2012). Distributed computation: the new wave of synthetic biology devices. Trends in biotechnology, 30(6):342-349.

Morin, P. J. (2009). Community ecology. John Wiley \& Sons.
Pickering, A. (2024). What is agency? a view from science studies and cybernetics. Biological Theory, 19(1):16-21.

Song, H., Payne, S., Gray, M., and You, L. (2009). Spatiotemporal modulation of biodiversity in a synthetic chemical-mediated ecosystem. Nature chemical biology, 5(12):929-935.

Tagkopoulos, I., Liu, Y.-C., and Tavazoie, S. (2008). Predictive behavior within microbial genetic networks. science, 320(5881):1313-1317.

Tamsir, A., Tabor, J. J., and Voigt, C. A. (2011). Robust multicellular computing using genetically encoded nor gates and chemical 'wires'. Nature, 469(7329):212-215.

# Inferring Spatial Dynamics through Interpretable Neural Cellular Automata 

Jakob H. Schauser ${ }^{1}$, Ala Trusina ${ }^{1}$<br>${ }^{1}$ Niels Bohr Institute, Denmark jakob.schauser@nbi.ku.dk


#### Abstract

Morphogenetic patterning can be reliably modeled using the self-organizing capabilities of Neural Cellular Automata (NCA). But given that we can emulate a dynamical system, what can we do to interpret it? Taking our onset in a specific biological model-system, we propose a method where regulatory elements driving the dynamics can be inferred by probing the internal interactions trough estimation of the Jacobian. Learning the emergent rules behind the NCA's allows us to better understand and engineer biological patterning.


## Submission type: Late Breaking Abstracts

## Graphical Abstract

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-192.jpg?height=211&width=828&top_left_y=1539&top_left_x=155)

## Introduction

Patterning is ubiquitous in nature. Everyone knows the characteristic Turing patterns visible on a zebra, but inarguably more important is the invisible patterning that defines and stabilizes cell fate decisions, driving embryonic development. This Morphogenetic patterning is determined by the interaction between a number of chemicals called morphogens, whose spatial organization is fundamental for morphogenesis, and, by extension, multicellular life itself.

There are infinitely many interactions that can lead to the same outcome, making solving the inverse problem both computationally and conceptually challenging. The purely local interactions of the NCA allow for modeling the dynamics while respecting the inherent spatial symmetries. But how can we pry open the black box?

In this work we propose a novel Jacobian-based interpretability method that allows for estimation of the interand intra-cellular dynamics that can lead to certain spatial distributions of cell types. Specifically, we are interested looking at morphogens in two-dimensional mouse gastruloid - a model system for morphogenetic patterning. Our method allows for finding intuitive and transparent ways to model the system. This can help both design experimental setups, run simulations, and possibly aid our understanding of certain biochemical processes.

## From Neural Self-Organization to interpretable dynamics

We present a brief outline of the proposed methodology:

1. Patterning is observed in the system
2. NCAs reproduce the pattern and give us the Jacobian
3. The Jacobian helps us infer the interactions in terms of a number of chemicals and their interactions
4. This understanding guides new simulation or experiments

Now, the specifics:
For two chemicals X and Y , to first order, their internal dynamics can be described by the Jacobian:

$$
\mathbf{J}=\left(\begin{array}{ll}
J_{X}^{X} & J_{Y}^{X} \\
J_{X}^{Y} & J_{Y}^{Y}
\end{array}\right)=\left(\begin{array}{ll}
\partial_{X} \dot{X} & \partial_{X} \dot{Y} \\
\partial_{Y} \dot{X} & \partial_{Y} \dot{Y}
\end{array}\right)
$$

Where $\dot{X}$ is $\frac{d X(X, Y, y)}{d t}$ and $\partial_{X} Y$ is shorthand for $\frac{\partial X}{\partial Y}$
The sign, magnitude, and dependencies of the entries in the Jacobian reveal the temporal and spatial interplay of the system variables, forming an interaction network similar to a gene regulatory network. Up- and down-regulation between neighboring cells can of course be direct reactions, but can also arise from diffusion processes, as shown in Equation 2 for a chemical $Y$ at point $i$ and neighboring points $j$ :

$$
\nabla^{2} Y_{i} \approx \frac{\sum_{j} Y_{j}-Y_{i}}{\Delta x} \propto J_{Y_{n b}}^{Y}-J_{Y}^{Y}
$$

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-193.jpg?height=426&width=841&top_left_y=264&top_left_x=155)
Figure 1: (a) An example of an interaction network that corresponds to diffusion (b) An example of interaction networks for two cross-inhibiting, diffusing chemicals as inferred by the proposed method.
$X$ and $Y$ are the active variables, $B$ is the system boundary, and the nb-subscript corresponds neighbors. Red and green arrows are inhibition and promotion, respectively.

By such mapping we can begin to catalog the effective building blocks of the emergent patterning, including diffusion, cross-inhibition, self-activation, or long-range feedback. The ability to read these directly from an NCA-trained Jacobian means that otherwise opaque selforganizing systems can be translated into interpretable causal diagrams. The previously mentioned diffusion (Eq. 2) corresponds to the interaction network as seen in Figure 1a

## Two-dimensional patterning

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-193.jpg?height=351&width=337&top_left_y=1612&top_left_x=406)
Figure 2: The 2D gastruloid model system (adapted from Chen et al. (2024))

For method development, we took our onset in gastruloids as described in Siggia and Warmflash (2018) for example. As a model system, the two-dimensional gastruloid provides a minimal and controllable setup for studying gastrulation patterning in vitro, were especially a number of stable, concentric rings are of interest (as seen in Figure 2).

Our lab has recently developed an in silico agent-based model, explaining some of the diverse dynamics observed in the system. We utilized this data to develop the approach presented here.

## Preliminary Results / Discussion

For simple patterning, NCAs quickly learn stable, reproducible update rules. The inferred interaction networks imply biologically plausible causal relationships, with realistic diffusion, inhibition, and promotion.

As an example, inferring the dynamics for two selfinhibiting chemicals with identical boundary conditions result in the interaction network as seen in Figure 1b. This corresponds to the following short-hand equations:

$$
\begin{aligned}
& \dot{X}_{i} \propto B_{i}+Y_{i}-X_{i}+\sum_{j}^{\{n b s\}}\left(B_{j}+Y_{j}-X_{j}\right) \\
& \dot{Y}_{i} \propto X_{i}-Y_{i}+\sum_{j}^{\{n b s\}}\left(X_{j}-Y_{j}\right)
\end{aligned}
$$

When simulating the above equations (after tuning relevant parameters), the resulting patterning correctly resemble the two observed chemicals.

More complex dynamics also show promising results, though further validation is ongoing. For most simple observed patterns, sensible interaction networks are generated, but interpretability depends on the linearity and separability of the active variables; nonlinearities (and-logic) remain difficult to disentangle.

The methodology is flexible enough for either training the NCA to reproduce the end-result or the full time-series, depending on the experimental constraints. When observing fewer time steps, the possible solution space increases, leading to a broader variety of discovered solutions. We are currently investigating how to best/most exhaustively search this space, a as well as exploring the generalization of our approach to other systems.

## Final remarks

In this work we have presented a framework for interpreting self-organizing solutions through NCAs. The method was applied to a well known model system and while in a very early stage, shows promising result. Our approach, focusing on interpretability, generates hypotheses, which can ultimately be tested experimentally.

By bridging black-box learning and interpretable modeling, Neural Cellular Automata can provide a powerful tool for exploring emergent patterning, allowing us to further our understanding of the chemical basis of morphogenesis.

## Acknowledgements

This work was supported by the Novo Nordisk Foundation, grant No. NNF23OC0086722

## References

Chen, B., Khan, H., Yu, Z., Yao, L., Freeburne, E., Jo, K., Johnson, C., and Heemskerk, I. (2024). Extended culture of 2d gastruloids to model human mesoderm development. undefined.

Siggia, E. D. and Warmflash, A. (2018). Modeling mammalian gastrulation with embryonic stem cells. Current Topics in Developmental Biology, pages 1-23.

# Robots and Time: Synthesizing Timescales for Embodied Artificial Systems 

Suet Lee, ${ }^{1 \S}$ and Heiko Hamann ${ }^{2 \S}$<br>${ }^{§}$ University of Konstanz, Konstanz, Germany and Centre for the Advanced Study of Collective Behaviour (CASCB), University of Konstanz, Konstanz, Germany<br>Email: ${ }^{1}$ suet.lee@uni-konstanz.de, ${ }^{2}$ heiko.hamann@uni-konstanz.de


#### Abstract

Time plays a fundamental yet often underappreciated role in the design, control, and understanding of robotic systems. From reflex-like motor responses occurring in milliseconds, to long-term learning and adaptation processes unfolding over days or weeks, robots operate across a hierarchy of interacting temporal scales. However, in comparison to biological organisms and systems, robots remain extremely limited in their ability to leverage timescales and to perceive time. Understanding and designing explicitly for such temporal dynamics will be crucial for robust and adaptive robotic intelligence. From bio-inspiration to operationalization, we review how timescales are integrated in biology, and we identify five key design principles for timescale synthesis in robot systems. We propose that the holistic integration of relevant temporal factors will be key to designing truly embodied systems, capable of safe and robust behavior and long-term autonomy. To this end, we identify the tools, metrics and architectures which will be needed for such integration.


## Submission type: Late Breaking Abstract

## From Biological to Artificial Timescales

All complex systems, whether biological or artificial, operate on multiple timescales. Biological organisms may operate on timescales ranging from microseconds, milliseconds, seconds, to circadian rhythms (Buonomano, 2007). Biological systems span even broader timescales, ranging from seconds to millennia in ecology (Behavioral Ecology, Paleoecology) and from days to $10^{7}$ years in biological evolution (experimental evolution, speciation). For collectives, group dynamics may manifest through short-term minute-to-minute interactions as well as day-to-day decisions - for example, in the formation of social groups through fissionfusion dynamics (Madsen and de Silva, 2024). Animals make use of a diverse set of mechanisms for perceiving and extracting temporal information. Such mechanisms enable the ability to "organize their functions, behaviors and $\operatorname{cog}$ nitive abilities according to temporal information" (Basgol et al., 2021). Human learning and thought also operate on disparate timescales (Kahneman, 2011). The ability to understand or measure time is "a significant advantage for an

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-196.jpg?height=379&width=839&top_left_y=841&top_left_x=1084)
Figure 1: Visualization of biological (a)-(e) and artificial (f)-(i) timescales: (a) and (b) represent learning in the brain from memory and via Hebbian/homeostatic plasticity; (c) mammals synchronize to circadian rhythm and have diverse mechanisms for measuring time; (d) migration in birds; (e) ecological regime shifts; (f) neural networks learn over varying timescales from historical data; (g) drones operate with super-fast reaction times; ( $h$ ) robot collectives synchronizing; (i) robot planning over a longer time period. Artificial systems are currently more concentrated at faster timescales, whilst biological systems span a wider range.

organism. It allows for the timely reaction to regular or predicted events, reducing the pressure for fast processing of sensory input" (Robu et al., 2019).

Timescales, therefore, are not just implementation details but can be fundamental design elements in robotics. The timescales relevant to a robot system depend largely on its function and operational environment. Robots operate in a broad spectrum of environments and tasks that vary along multiple axes, including degree of uncertainty, safetycriticality, number of resource constraints, duration of operation, and the degree of dynamism or time-varying nature of the environment. Engineering often focuses too much on "faster is better" and assumes steady-state behavior. Current robot designs exploit multiple interacting timescales in a limited way only, for example, as a fixed hierarchy of nested control loops (e.g., inner loop for velocity control, outer loop for robot pose). Applications of control theory today still need to push technological boundaries (e.g., fast moving quadrotors) to increase sensor sampling and computing frequency (i.e., decreasing perception latency) in constant
struggle with latencies due to sensing and compute (Falanga et al., 2019). But natural systems often benefit from multiple sampling rates, selective attention, and timescale-matching, demonstrating remarkable properties of adaptability under high resource constraints. What if "intelligence" comes from matching the timescale of action and perception, not brute-force speed? The ability to identify relevant timescales for designing a robot system, according to its operational environment and function, has the potential to improve robustness, adaptivity and performance. Figure 1 compares the timescales present in biological and artificial systems.

## Operationalizing Timescales in Robots

Our interest is in synthesizing timescales in order to design robust robotic systems capable of long-term operation in complex environments. There are multiple reasons to focus on temporal information, including embodiment, long-duration operation, efficient use of the environment, and matching sensing, actuation, and processing to relevant timescales. The following is a summary of five key design principles for timescale synthesis in robot systems:

1) Slow and fast learning: Structuring learning across different temporal layers mirrors strategies observed in biological systems, and can also help in mitigating catastrophic forgetting and poor generalization (Parisi et al., 2019; Lesort et al., 2020). For robots with limited onboard compute and memory, important factors are the sampling rate of data (e.g., from sensors) and rates of data processing (e.g., learning rates). The influence of lifetime-vs-evolutionary timescales is also studied in artificial evolution and shows potential for open-ended systems (Nolfi et al., 1994; Doncieux et al., 2015; Soltoggio et al., 2018).
2) Scaling up predictions: Predictions are limited by onboard compute and sensing/reaction rate. Nexting and Active Inference approaches, together with an understanding of the relevant timescales, could allow robots to massively scale up predictions for salient metrics, enabling better performance through error estimation and action selection (Modayil et al., 2014; Murata et al., 2015; Lanillos et al., 2021). Anticipatory perception may enable robots to bypass computationally-heavy predictions completely via heuristics (Datteri et al., 2003).
3) Timescales for collective timing decisions: The problem of choosing the right time for the collective to take action is a gap in research (Kao et al., 2024). Indeed, embedding multiple timescales in collective robotic systems may enable more sophisticated coordination via collective timing decisions (Degesys et al., 2007; O' Keeffe and Bettstetter, 2019). For decision-making processes in robot collectives, it has been shown that slower processes can sometimes outperform faster ones (Valentini et al., 2016; Aust et al., 2022; Reina et al., 2024). The optimal rate for such processes is therefore not obvious.
4) Layered timescale integration: Layers encode multiple
timescales with feedback loops between layers for integration. Layers may emerge by imposing "bottlenecks" on a neural network (Paine and Tani, 2005; Yamashita and Tani, 2008; Jian et al., 2023), or may be explicitly encoded, as with a Layered Control Architecture (Matni et al., 2024) or time-varying activation functions (Nolfi, 2002). Blockchain smart contracts have been identified as an emerging research direction for hierarchical control of decision-making processes - this approach can enable different rates of (fastslow) decision-making in multi-robot systems (Dorigo et al., 2024).
5) Time perception capabilities: Comparison with time perception capabilities in biological cognition highlights the missing capabilities in robots - designing for these capabilities may provide the basis for new frameworks and approaches for operationalizing timescales (Basgol et al., 2021; Maniadakis and Trahanias, 2014).

## Discussion

A holistic integration of relevant temporal factors is missing from current robotics research, which may be key to shift towards truly embodied systems (Sünderhauf et al., 2018). Three practical elements that will be necessary to operationalize timescales for real-world deployment include:

1) Tools for timescale discovery: fast-slow dynamics have been identified in many complex and self-organizing systems, but such timescales are not always explicit nor wellseparated. An iterative design process for timescale discovery would enable "matching" for relevant timescales.
2) Metrics for temporal robustness: "timescale alignment tests" can ensure that robot control remains aligned with the relevant timescales of the system. Metrics can determine the timescale alignment of a system over time (e.g., using frequency characteristics (Ye et al., 2025)). Timescale separation and nontemporal sequences are also useful variables for measurement (Iida and Giardina, 2023).
3) Self-adaptive timescale usage via dynamic control architectures: Real-time adaptation is essential for dynamic environments and systems that undergo change. In particular, degradation and the occurrence of faults are inevitable during operation. The relevant timescales may change over time as a result - e.g. if a robot's ability to maneuver itself is impaired or if maximum velocity decreases, it may have slower response times and a high sensing rate might be inefficient.

By integrating tools, metrics, and adaptive control architectures, robotics can operationalize timescales as a firstclass design principle, and this capacity to discover, measure, and dynamically adapt them may define resilient realworld robotics.

## Acknowledgements

This work has been partially supported by the DFG under Germany's Excellence Strategy, EXC 2117, 422037984.

## References

Aust, T., Talamali, M. S., Dorigo, M., Hamann, H., and Reina, A. (2022). The hidden benefits of limited communication and slow sensing in collective monitoring of dynamic environments. In International Conference on Swarm Intelligence, pages 234-247. Springer.

Basgol, H., Ayhan, I., and Ugur, E. (2021). Time perception: A review on psychological, computational, and robotic models. IEEE Transactions on Cognitive and Developmental Systems, 14(2):301-315.

Buonomano, D. V. (2007). The biology of time across different scales. Nature Chemical Biology, 3(10):594-597.

Datteri, E., Teti, G., Laschi, C., Tamburrini, G., Dario, G., and Guglielmelli, E. (2003). Expected perception: an anticipation-based perception-action scheme in robots. In Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), volume 1, pages 934-939. IEEE.

Degesys, J., Rose, I., Patel, A., and Nagpal, R. (2007). Desync: Self-organizing desynchronization and TDMA on wireless sensor networks. In Proceedings of the 6th international conference on Information processing in sensor networks, pages 11-20.

Doncieux, S., Bredeche, N., Mouret, J.-B., and Eiben, A. E. (2015). Evolutionary robotics: what, why, and where to. Frontiers in Robotics and AI, 2:4.

Dorigo, M., Pacheco, A., Reina, A., and Strobel, V. (2024). Blockchain technology for mobile multi-robot systems. Nature Reviews Electrical Engineering, 1(4):264-274.

Falanga, D., Kim, S., and Scaramuzza, D. (2019). How fast is too fast? The role of perception latency in high-speed sense and avoid. IEEE Robotics and Automation Letters, 4(2):18841891.

Iida, F. and Giardina, F. (2023). On the timescales of embodied intelligence for autonomous adaptive systems. Annual Review of Control, Robotics, and Autonomous Systems, 6(1):95-122.

Jian, N. L., Zabiri, H., and Ramasamy, M. (2023). Control of the multi-timescale process using multiple timescale recurrent neural network-based model predictive control. Industrial \& Engineering Chemistry Research, 62(15):6176-6195.

Kahneman, D. (2011). Thinking, fast and slow. Macmillan.
Kao, A. B., Banerjee, S. C., Francisco, F. A., and Berdahl, A. M. (2024). Timing decisions as the next frontier for collective intelligence. Trends in Ecology \& Evolution.

Lanillos, P., Meo, C., Pezzato, C., Meera, A. A., Baioumy, M., Ohata, W., Tschantz, A., Millidge, B., Wisse, M., Buckley, C. L., et al. (2021). Active inference in robotics and artificial agents: Survey and challenges. arXiv preprint arXiv:2112.01871.

Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., and Díaz-Rodríguez, N. (2020). Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. Information Fusion, 58:52-68.

Madsen, A. and de Silva, S. (2024). Societies with fission-fusion dynamics as complex adaptive systems: the importance of scale. Philosophical Transactions B, 379(1909):20230175.

Maniadakis, M. and Trahanias, P. (2014). Time in consciousness, memory and human-robot interaction. In International Conference on Simulation of Adaptive Behavior, pages 11-20. Springer.

Matni, N., Ames, A. D., and Doyle, J. C. (2024). Towards a theory of control architecture: A quantitative framework for layered multi-rate control. arXiv preprint arXiv:2401.15185.

Modayil, J., White, A., and Sutton, R. S. (2014). Multi-timescale nexting in a reinforcement learning robot. Adaptive Behavior, 22(2):146-160.

Murata, S., Yamashita, Y., Arie, H., Ogata, T., Sugano, S., and Tani, J. (2015). Learning to perceive the world as probabilistic or deterministic via interaction with others: A neurorobotics experiment. IEEE Transactions on Neural Networks and Learning Systems, 28(4):830-848.

Nolfi, S. (2002). Evolving robots able to self-localize in the environment: the importance of viewing cognition as the result of processes occurring at different time-scales. Connection Science, 14(3):231-244.

Nolfi, S., Floreano, D., Miglino, O., Mondada, F., et al. (1994). How to evolve autonomous robots: Different approaches in evolutionary robotics. In Artificial Life IV, pages 190-197.

O'Keeffe, K. and Bettstetter, C. (2019). A review of swarmalators and their potential in bio-inspired computing. Microand Nanotechnology Sensors, Systems, and Applications XI, 10982:383-394.

Paine, R. W. and Tani, J. (2005). How hierarchical control selforganizes in artificial adaptive systems. Adaptive Behavior, 13(3):211-225.

Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., and Wermter, S. (2019). Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71.

Reina, A., Njougouo, T., Tuci, E., and Carletti, T. (2024). Speedaccuracy trade-offs in best-of-n collective decision making through heterogeneous mean-field modeling. Physical Review $E$, 109(5):054307.

Robu, A. D., Salge, C., Nehaniv, C. L., and Polani, D. (2019). Measuring time with minimal clocks. Artificial Life, 25(4):383409.

Soltoggio, A., Stanley, K. O., and Risi, S. (2018). Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks. Neural Networks, 108:48-67.

Sünderhauf, N., Brock, O., Scheirer, W., Hadsell, R., Fox, D., Leitner, J., Upcroft, B., Abbeel, P., Burgard, W., Milford, M., et al. (2018). The limits and potentials of deep learning for robotics. The International journal of robotics research, 37(4-5):405-420.

Valentini, G., Ferrante, E., Hamann, H., and Dorigo, M. (2016). Collective decision with 100 kilobots: Speed versus accuracy in binary discrimination problems. Autonomous agents and multi-agent systems, 30:553-580.

Yamashita, Y. and Tani, J. (2008). Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Computational Biology, 4(11):e1000220.

Ye, F., Abdulali, A., Chu, K.-F., Zhang, X., and Iida, F. (2025). Reservoir controllers design though robot-reservoir timescale alignment. Communications Engineering, 4(1):81.

# Neural Particle Automata: Learning Self-Organizing Particle Systems 

Hyunsoo Kim ${ }^{* 1}$, Ehsan Pajouheshgar*2, Sabine Süsstrunk ${ }^{2}$, and Jinah Park ${ }^{1}$<br>${ }^{1}$ KAIST, South Korea $\quad{ }^{2}$ EPFL, Switzerland<br>*These authors contributed equally to this work.<br>\{khskhs, jinahpark\}@kaist.ac.kr, \{ehsan.pajouheshgar, sabine.susstrunk\}@epfl.ch


#### Abstract

We propose Neural Particle Automata (NPAs), a generalization of Neural Cellular Automata (NCAs) for dynamic particle systems, rather than static lattices. In NPAs, each particle carries an internal state and a position, updating both through trainable local rules informed by differentiable Smoothed Particle Hydrodynamics (SPH) perception. Our framework extends the expressivity of NCAs to particle systems, enabling the discovery of self-organizing particle dynamics that create a diverse set of patterns and morphologies.


## Submission type: Late Breaking Abstracts

## Introduction

Self-organizing systems emerge from simple agents interacting locally, which produce complex global patterns observed in physics, biology, and sociology. Early computational models, such as cellular automata and reactiondiffusion systems, captured such emergence through handcrafted rules. Neural Cellular Automata (NCAs) (Mordvintsev et al., 2020) replace these rules with trainable neural networks that evolve the internal state of each cell. NCAs exhibit emergent phenomena such as regeneration and robustness, while also enabling goal-driven, dataguided pattern modeling. Since their introduction, NCAs have been adapted to different spatial structures, including graphs (Grattarola et al., 2021), meshes (Pajouheshgar et al., 2024a), and arbitrary irregular lattices (Kim and Park, 2025).

Despite this progress, all existing NCAs share a fundamental limitation: cells are pinned to a fixed lattice. Real-world agents are inherently particle-like: they migrate, swarm, and respond to chemical gradients, constantly shifting patterns of local connectivity. Classical latticebased systems such as Game of Life (Gardner, 1970), Lenia (Chan, 2020), and NCAs cannot capture this dynamics of motion. Meanwhile, particle-based systems like Clusters (Ventrella, 2016), Particle Life (Mohr, 2022), and Particle-Lenia (Mordvintsev et al., 2022) exhibit rich emergent dynamics but lack trainability from data and the flexibility of neural models.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-200.jpg?height=353&width=844&top_left_y=833&top_left_x=1078)
Figure 1: Perception mechanism in Neural Particle Automata. (Left) An example of NPA trained to organize a gecko shape. (Right) SPH-based neighborhood perception.

In this work, we propose Neural Particle Automata (NPAs), a generalization of NCAs to dynamic particle systems. In NPAs, each particle carries both an internal state and a position in continuous space, updated by a trainable neural network. Robust local perception is achieved by differentiable, mesh-free operators from Smoothed Particle Hydrodynamics (SPH), as illustrated in Figure 1, which estimates neighborhood features and spatial gradients from particle samples.

Preliminary experiments demonstrate that NPAs inherit the hallmark properties of lattice-based NCAs such as robustness, regeneration, and emergent pattern formation. They also exhibit fundamentally new dynamics enabled by particle motion, such as swarming, collective flows, and morphology formation, that were not possible in prior NCA frameworks.

## Method

Our work builds on the SPH-NCA framework proposed by Kim and Park (2025), which extended NCAs to irregular point samplings using SPH method. In their formulation, each point is represented by its position $x_{i} \in \mathbb{R}^{d}$ and state vector $S_{i} \in \mathbb{R}^{C}$. In conventional NCAs, at each update step, a cell $i$ perceives the local neighborhood with spatial gradient estimators such as Sobel or Laplacian operators, which are proven highly effective for NCAs to learn robust and scale-invariant behavior (Pajouheshgar et al., 2024b). To ex-
tend to irregular point samplings, SPH-NCA uses differentiable SPH gradient operator to perceive the neighborhood, which can be written as:

$$
\Delta S_{i}=f_{\theta}\left(S_{i}, \nabla S_{i}\right),
$$

where $f_{\theta}$ is a Multi-Layer Perceptron (MLP) with parameters $\theta$, and $\nabla S_{i}$ is the SPH gradient of the states at $x_{i}$.

We extend this formulation to enable the evolution of $d y$ namic particles, where each step updates not only the particle's internal state $S_{i}$ but also its position $x_{i}$. The new update rule takes the form:

$$
\left[\Delta x_{i}, \Delta S_{i}\right]=f_{\theta}\left(S_{i}, \tilde{S}_{i}, \nabla S_{i}, \nabla \rho_{i}\right) .
$$

Here, $f_{\theta}$ predicts both state changes $\Delta S_{i}$ and displacement vectors $\Delta x_{i}$ from the perception vector ( $S_{i}, \tilde{S}_{i}, \nabla S_{i}, \nabla \rho_{i}$ ). $\tilde{S}_{i}$ denotes the SPH-smoothed average of neighboring states, while $\nabla S_{i}$ denotes the spatial gradient of the state, providing zeroth- and first-order approximations of the neighborhood, respectively. The particle density gradient $\nabla \rho_{i}$ further allows the network to model behaviors of particles that aggregate or spread apart depending on their internal state and perception. To compare the evolving particle system with a target image or pattern, we differentiably render the particle densities and RGB channnels from the state onto the grid using Gaussian splatting, enabling end-to-end training of our model with gradient-descent.

## Experiments

## Differentiable SPH operators

Training NPAs relies on differentiable SPH operators that allow backpropagation through multiple perception steps. We tested two different implementations:
![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-201.jpg?height=282&width=305&top_left_y=1448&top_left_x=680) index-based PyTorch functions with automatic differentiation, and custom hashgrid-based CUDA kernels for both forward and backward passes. We found that during training, custom kernels (orange) can scale linearly to the number of particles, while index-based implementation (blue) scales superlinearly.

Generating Morphologies and Textures We train our NPA models on two distinct tasks: growing morphologies and synthesizing textures. In both experiments, we use $C=16$ dimensional cell states, initialized with zero, and apply stabilization techniques from Mordvintsev et al. (2020), including a checkpoint pool and gradient normalization.

For the morphology task, we initialize 4096 particles with Gaussian-distributed positions. The model successfully generates and maintains the complex silhouette and colors of the gecko. As shown in Figure 2a, particles first establish coarse gradients aligned with the target's major axes, then progressively form recognizable body parts such as head, limbs, and tail-before converging to a stable mature state.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-201.jpg?height=1013&width=858&top_left_y=301&top_left_x=1064)
Figure 2: Summary of results. Videos are available here.

For the texture task, we initialize 16384 particles with uniformly random positions. The particles learn to selforganize to form coherent patterns and adapt their density according to local detail as shown in Figure 2b.

## Conclusion and Future Works

We introduce Neural Particle Automata (NPAs), a novel framework that extends NCAs to learn the evolving dynamics of particles, encompassing both their states and positions, directly from objective functions.

Our preliminary work demonstrates that NPAs inherit core NCA properties like self-regeneration, while uniquely exhibiting fundamentally new dynamics such as swarming, collective flows, and complex morphology formation.

We envision NPAs as a step toward trainable, particlebased self-organization models that combine the life-like richness of agent-based systems with the learnability and efficiency of NCAs. Future extensions could further enable richer particle dynamics, such as co-existing NPA species with distinct rule sets, as well as mechanisms for dynamic particle division and removal.

## Acknowledgements

This research was supported by KAIST Venture Research Program for Graduate \& Ph.D students.

## References

Chan, B. W.-C. (2020). Lenia and Expanded Universe. volume ALIFE 2020: The 2020 Conference on Artificial Life of Artificial Life Conference Proceedings, pages 221-229.

Gardner, M. (1970). Mathematical games: The fantastic combinations of john conway's new solitaire game "life". Scientific American, 223(4):120.

Grattarola, D., Livi, L., and Alippi, C. (2021). Learning graph cellular automata. Advances in Neural Information Processing Systems, 34:20983-20994.

Kim, H. and Park, J. (2025). Train once, generate anywhere: Discretization agnostic neural cellular automata using sph method. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters, pages 1-2.

Mohr, T. (2022). Particle life simulator. http:// particle-life.com/. Open-source Java simulation of emergent particle life, using parallel processing and space partitioning; includes GitHub, YouTube, and Discord components.

Mordvintsev, A., Niklasson, E., and Randazzo, E. (2022). Particle lenia and the energy-based formulation. https://google-research.github.io/ self-organising-systems/particle-lenia/. Accessed: 2025-09-08.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing neural cellular automata. Distill, 5(2):e23.

Pajouheshgar, E., Xu, Y., Mordvintsev, A., Niklasson, E., Zhang, T., and Süsstrunk, S. (2024a). Mesh neural cellular automata. ACM Transactions on Graphics (TOG), 43(4):1-16.

Pajouheshgar, E., Xu, Y., and Süsstrunk, S. (2024b). Noisenca: Noisy seed improves spatio-temporal continuity of neural cellular automata. volume ALIFE 2024: Proceedings of the 2024 Artificial Life Conference of Artificial Life Conference Proceedings, page 57.

Ventrella, J. (2016). Clusters. https://ventrella.com/ Clusters/intro.html. Accessed: 2025-09-08.

# Collective Behavior Emerging from Social Learning Strategies and Network Structures 

Jingyu Xi ${ }^{1}$, Wataru Toyokawa ${ }^{2}$<br>${ }^{1}$ University of Konstanz, Germany<br>${ }^{2}$ Computational Group Dynamics (COGNAC) Collaboration Unit at RIKEN CBS, Japan wataru.toyokawa@riken.jp


#### Abstract

Humans make decisions collectively by combining individual and social learning. Individuals benefit from groups when individual exploration fails to accurately assess the environment, a phenomenon known as the "wisdom of crowds." Previous studies indicate that self-organizing group dynamics can reduce suboptimal biases in noisy environments, particularly in fully connected groups. However, agents often have only partial information due to cognitive and physical constraints. To explore how diverse social network structures influence the collective dynamics of social learners, we integrate a decentralized network with a social reinforcement learning model in repeated two-armed bandit tasks. Our results suggest that: 1 . Social learning in a sparse network outperforms asocial solo learning in highly uncertain tasks. 2. The phenomenon "less is more, and more is different" holds true only when agents strategically balance individual and social learning. 3. The group size effect on collective performance is significantly influenced by network structures.


## Submission type: Late Breaking Abstracts

## Introduction

Understanding the principles of collective behavior has been an enduring conundrum in behavioral sciences. Previous work suggests that conformist social learning, whereby the most common behavior in a group is disproportionately more likely to be copied, can regulate both the wisdom and "madness" of interactive human groups (Toyokawa et al., 2019; Toyokawa and Gaissmaier, 2022). By combining reinforcement learning with a decision-biasing imitation process (Najar et al., 2019), agent-based model simulations have identified a critical level of conformity bias, above which collective behavior becomes dominated by herding, resulting in inflexible collective decision-making. On the other hand, collective learners can remain flexible as long as reliance on the social information remains sufficiently low (Lorenz et al., 2011; Toyokawa et al., 2019). These studies suggest that the conflict between collective intelligence and maladaptive herding could potentially be predicted with quantitative knowledge of the social learning strategies employed.

Previous studies on cooperation (Rand et al., 2014; Shirado and Christakis, 2020), social foraging (Cantor and Farine, 2018), coordination (Shirado and Christakis, 2017; Angus and Newton, 2020), problem-solving in a complex landscape (Lazer and Friedman, 2007), and cultural evolution (Smolla and Akçay, 2019; Cantor et al., 2021) have shown that information transfer across structured social connections can qualitatively alter the outcome of collective behavior, compared to the pattern observed in populations where individuals are randomly or fully connected. By exploring a range of network density, previous work has generally concluded that the denser the network, the higher the risk of herding due to increased correlations of opinions between neighboring agents (Reina et al., 2024; Dasaratha and He , 2020). Similarly to observations in conformist social learning studies, the conflict between collective intelligence and maladaptive herding appears to be predictable with quantitative knowledge of the realized social network structure.

However, the decision-making processes assumed in these social network studies were often overly simplified, where individuals merely made a one-shot decision based on provided social cues and, therefore, did not actively engage in self-evaluated trial-and-error individual learning during social interactions (Das et al., 2014; Brackbill and Centola, 2020; Lazer and Friedman, 2007). For example, in collective behavior studies with the NK model, there is usually no learning per se, but instead more emphasis on search and selection (Lazer and Friedman, 2007). As a result, it remains unclear how diverse social structures interact with the learning strategies deployed. Since a lack of individual exploration often causes social learners to become stuck in suboptimal decision-making within fully connected networks (Toyokawa and Gaissmaier, 2022), it is possible that certain types of less-connected networks could "rescue" clusters of conformist learners by facilitating the flow of new information from other parts of the network. On the other hand, the risk of herding in dense networks might be mitigated by the presence of more explorative individuals.

The key questions we aim to address here are threefold:

- How does collective decision-making performance, compared to both solo learning and fully connected social learning, benefit from less dense social connections?
- What kinds of nonlinear collective decision phenomena emerge in both risky and non-risky decision environments?
- How does individual-level decision performance influence emergent collective behavior?


## Methods

## The Social Two-Armed Bandit Task

To address these questions, we employed a social "twoarmed bandit" task, in which individuals could observe others' decision-making behavior (social frequency information) but not their payoff information. Unlike the previous "fully-connected" studies (Toyokawa et al., 2019; Toyokawa and Gaissmaier, 2022), social information was available only from predetermined social connections. To manipulate individual exploration and exploitation patterns, and to potentially impact the resulting collective behavior (Toyokawa and Gaissmaier, 2022), we tested two different reward distributions. In the risk-homogeneous setting, the variance of payoffs from both behavioral options was equal, whereas in the risk-heterogeneous setting, the variance of payoffs from one option (the riskier option) was higher than that of the other option (the safer option). We set the expected payoff of the riskier option to always be higher than that of the safer option. It is known that individual trial-and-error learning tends to result in avoidance of such a riskier option, even if it provides higher expected payoffs in the long run, due to biased exploration (a phenomenon known as the hot stove effect; Denrell and March (2001); Denrell (2007, 2024)). Therefore, the risk-heterogeneous setting allowed us to examine how such biased decision-making operating at the individual level might interact with both social learning and network structure.

## Network Manipulations

We manipulated network density using the " 2 m -ring" network algorithm to explore the importance of a more distributed system without central individuals (or "nodes" of the social network). The 2 m -ring network algorithm generates networks in which each node is connected to its $m$ nearest neighbors on both the "left-hand side" and the "righthand side". This approach ensures that all nodes have the same number of connections. Denoting the number of connections each node has as $\langle k\rangle$, which is called "degree", then each node has degree $\langle k\rangle=2 m$. We assumed that the network structure does not change over the course of the bandit task, and that social information transmits in both directions between connected nodes.

Previous studies on network dynamics and the wisdom of crowds (Becker et al., 2017) demonstrate that decentralized
systems, where individuals have equal social influences, result in higher accuracy in a group task compared to centralized networks. Popular network models such as the scalefree model (Li et al., 2007) and the random network model (Barabâsi et al., 2002) that have been widely tested in network science and information diffusion studies, are great abstractions representing real-world complex networks. However, due to the randomness and noise those models naturally possess, the results might be difficult to disentangle and interpret if the networks generated by these algorithms are centralized. Thus our work uses the more decentralized " 2 m -ring" network to explore the mechanism between collective learning and network features.

## Results and Discussion

In this work, we integrated social reinforcement learning with network features to explore how the explo-ration-exploitation trade-off in individual learning, combined with frequency-dependent social learning strategies at the individual level and information structure at the collective level, shapes collective learning dynamics. Our simulations demonstrate that groups generally benefit from the "wisdom of crowds" when the environment is highly uncertain and risky. We also showed that collective dynamics can benefit from reduced social information in both non-risky and risky environments, even though animals-including humans-tend to actively seek social information when uncertain.

Furthermore, we highlight the similar functional roles of learning parameters at the individual level and network features at the system level, with potential implications for decision-making interventions that operate across scales. This similarity can be explained by a form of positive feedback that emerges at both levels. At the individual level, increasing the values of social learning parameters reduces the benefits of stochastic exploration and amplifies the influence of social information. At the system level, increasing network density gives each agent access to more social informants, leading to more accurate sampling of social information across the population. If agents happen to discover the optimal solution early, the system can rapidly converge on it. However, if agents become stuck in suboptimal options, high social learning values and dense connectivity hinder the stochastic variability that might otherwise help the group escape from this pitfall.

This study marks the beginning of an exploration into how information structures influence the emergence of collective intelligence from self-organizing individual behavior, using a reinforcement learning framework. Many network features that characterize real-world social interactions remain to be examined. Nevertheless, this study represents one of the first attempts to integrate cross-scale parameters into learning models and reveals novel patterns that were not apparent in existing theories.

## Acknowledgements

This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy.

## References

Angus, S. D. and Newton, J. (2020). Collaboration leads to cooperation on sparse networks. PLoS Computational Biology, 16(1):e1007557.

Barabâsi, A.-L., Jeong, H., Néda, Z., Ravasz, E., Schubert, A., and Vicsek, T. (2002). Evolution of the social network of scientific collaborations. Physica A: Statistical mechanics and its applications, 311(3-4):590-614.

Becker, J., Brackbill, D., and Centola, D. (2017). Network dynamics of social influence in the wisdom of crowds. Proceedings of the national academy of sciences, 114(26):E5070-E5076.

Brackbill, D. and Centola, D. (2020). Impact of network structure on collective learning: An experimental study in a data science competition. PLoS One, 15(9):e0237978.

Cantor, M., Chimento, M., Smeele, S. Q., He, P., Papageorgiou, D., Aplin, L. M., and Farine, D. R. (2021). Social network architecture and the tempo of cumulative cultural evolution. Proceedings of the Royal Society B, 288(1946):20203107.

Cantor, M. and Farine, D. R. (2018). Simple foraging rules in competitive environments can generate socially structured populations. Ecology and evolution, 8(10):4978-4991.

Das, A., Gollapudi, S., and Munagala, K. (2014). Modeling opinion dynamics in social networks. In Proceedings of the 7 th $A C M$ international conference on Web search and data mining, pages 403-412.

Dasaratha, K. and He, K. (2020). Network structure and naive sequential learning. Theoretical Economics, 15(2):415-444.

Denrell, J. (2007). Adaptive learning and risk taking. Psychological review, 114(1):177.

Denrell, J. (2024). Adaptive sampling policies imply biased beliefs: A generalization of the hot stove effect. arXiv preprint arXiv:2404.02591.

Denrell, J. and March, J. G. (2001). Adaptation as information restriction: The hot stove effect. Organization science, 12(5):523-538.

Lazer, D. and Friedman, A. (2007). The network structure of exploration and exploitation. Administrative science quarterly, 52(4):667-694.

Li, W., Zhang, X., and Hu, G. (2007). How scale-free networks and large-scale collective cooperation emerge in complex homogeneous social systems. Physical Review E-Statistical, Nonlinear, and Soft Matter Physics, 76(4):045102.

Lorenz, J., Rauhut, H., Schweitzer, F., and Helbing, D. (2011). How social influence can undermine the wisdom of crowd effect. Proceedings of the national academy of sciences, 108(22):9020-9025.

Najar, A., Bonnet, E., Bahrami, B., and Palminteri, S. (2019). Imitation as a model-free process in human reinforcement learning. bioRxiv, page 797407.

Rand, D. G., Nowak, M. A., Fowler, J. H., and Christakis, N. A. (2014). Static network structure can stabilize human cooperation. Proceedings of the National Academy of Sciences, 111(48):17093-17098.

Reina, A., Njougouo, T., Tuci, E., and Carletti, T. (2024). Speedaccuracy trade-offs in best-of-n collective decision making through heterogeneous mean-field modeling. Physical Review E, 109(5):054307.

Shirado, H. and Christakis, N. A. (2017). Locally noisy autonomous agents improve global human coordination in network experiments. Nature, 545(7654):370-374.

Shirado, H. and Christakis, N. A. (2020). Network engineering using autonomous agents increases cooperation in human groups. Iscience, 23(9).

Smolla, M. and Akçay, E. (2019). Cultural selection shapes network structure. Science advances, 5(8):eaaw0609.

Toyokawa, W. and Gaissmaier, W. (2022). Conformist social learning leads to self-organised prevention against adverse bias in risky decision making. eLife, 11.

Toyokawa, W., Whalen, A., and Laland, K. N. (2019). Social learning strategies regulate the wisdom and madness of interactive crowds. Nature Human Behaviour, page 326637.

# A test-bed for open-ended evolution and skill acquisition for embodied collectives 

Max Oltmanns ${ }^{1}$, and Jonas Kuckling ${ }^{1,2,3}$<br>${ }^{1}$ University of Konstanz, Department of Computer and Information Science<br>${ }^{2}$ Zukunftskolleg, University of Konstanz<br>${ }^{3}$ Centre for the Advanced Study of Collective Behaviour, University of Konstanz<br>jonas.kuckling@uni-konstanz.de


#### Abstract

This extended abstract reports on our progress in an ongoing project. We study open-endedness in embodied collectives. To that end, we are currently developing a simulated test-bed that will allow virtual agents to continuously evolve and acquire new skills. We first present first the aims of the project and our motivation for creating the test-bed. We then present the current state of the test-bed. Lastly, we highlight the open challenges and remaining work in this project. We contend that research in open-endedness for embodied collectives will require the following three elements: i) modular architectures for controlling the agents; ii) a simulated, interaction rich environment; iii) measures of novelty, complexity and open-endedness. We have build a proof of concept testbed. The test-bed uses Minecraft as a virtual environment and the agents are controlled through behavior trees. As an initial benchmark, we have implemented a modified scenario of open-ended foraging (Witkowski and Ikegami, 2019). The current state of the framework serves as a basic implementation to investigate open-endedness in embodied collectives. Future work will be dedicated to enhancing the test-bed, implementing measures of open-endedness in collectives, and applying the framework to a swarm of mobile robots.


Submission type: Late Breaking Abstract

## Introduction

Embodied artifical intelligence is the extension of artificial intelligence principles to agents with bodies that are situated in the real world (Brooks, 1991; Pfeifer and Iida, 2004; Savva et al., 2019). However, manually designing software for these agents has proven often infeasible and automatic methods, such as reinforcement learning or (neuro)evolution, are often used (Kuckling, 2023). However, these methods are goal-directed, as they require an explicit objective function (also called "fitness" or "reward" function). Alternative approaches, such as imitation learning replace the explicit objective function with other representations of the desired behavior (Celemin et al., 2022; Kuckling, 2023). Yet, these techniques all require the existence of a target behavior. Contrarily, techniques such as novelty search (Lehman and Stanley, 2011) and quality-diversityalgorithms (Pugh et al., 2016) try to find a diverse set of
behaviors. While these methods have shown to find behaviors outperforming those designed through artificial evolution or reinforcement learning (Gaier et al., 2019), they are limited in the complexity of behaviors that can be found. Open-ended techniques promise to overcome this limitation by designing behaviors that are novel and of (potentially) unbounded complexity (Stanley, 2019; Stepney, 2021). Intelligence is, however, not only an individual trait. Many animals, such as ants or locusts, exhibit relatively limited apparent individual intelligence. But through cooperation in their collective, they are able to perform complex collective behaviors that appear to be more intelligent than what each individual is capable of (Bonabeau et al., 1999; Camazine et al., 2001). This emergence of collective behavior through self-organization can also be employed in artificial systems, such as robot swarms (Şahin, 2004; Brambilla et al., 2013).

This extended abstract reports on our progress in an ongoing project. We study open-endedness in embodied collectives. To that end, we are currently developing a simulated test-bed that will allow virtual agents to continuously evolve and acquire new skills.

## Project aims

We contend that research in open-endedness for embodied collectives will require the following three elements: i) modular architectures for controlling the agents; ii) a simulated, interaction rich environment; iii) measures of novelty, complexity and open-endedness.

Any fixed-size instance of control software would provide a fixed upper-limit to the complexity of behavior that could be achieved. It is therefore imperative, that, in order to create behavior of potentially unbounded complexity, the control software needs to be able to scale as well. We believe that modular control architectures are better suited to this task than architectures like neural networks. While neural networks can be grown during the open-ended process (Stanley and Miikkulainen, 2002), this growth has no inherent structure. Furthermore, neural networks are prone to catastrophic forgetting, so skills that have been acquired earlier might be completely lost at later times. In modular architectures,
conversely, prior learned skills can be saved in a repertoire and reused as a module in future, larger and more complex instances of control software. In particular, we focus on behavior trees (Colledanchise and Ögren, 2018), a modular architecture that has found many applications in robotics and artificial intelligence (Iovino et al., 2022). Besides their modularity, behavior trees are also inherently human readable and they offer two-way control transfers.

Besides the control software, it is also necessary that the environment is rich enough for meaningful complex behavior to emerge. For practical reasons, such as energy autonomy, it is necessary that the environment should be virtual. However, virtual environments provide an inherent upper limit to the complexity of the behavior that can be performed in them. It is therefore necessary to choose or create an environment that allows a great degree of freedom to the agents situated in it. Video games, such as Minecraft, are commonly used to this effect (Grbic et al., 2021; Altera et al., 2024; Wang et al., 2024; Dong et al., 2024) and we also focus on Minecraft as the environment for our agents.

Lastly, to decide if the process is truly open-ended (or to which degree we observe open-endedness), we need some ways to quantify these properties. We believe that there are three main components that are of high relevance: i) individual behavior; ii) state of the environment; iii) inter- and intragroup interactions. First, we can measure complexity and novelty of the behaviors of individual agents, comparing them against all previously encountered individual behaviors (Hughes et al., 2024). As the agents are situated in the environment and can manipulate it with their actions, we can also treat the change of the state of the environment as a "collective" action of all agents. Using similar approaches as for the individual behavior, we can quantify the degree of open-endedness resulting from the collective actions. Lastly, the agents can not only interact with the environment but also with each other. These intragroup interactions (for agents belonging to the same sub-group, if applicable) or intergroup interactions (for agents belonging to different sub-groups) will provide another degree of complexity that needs to be measured and quantified.

## Proof of concept

We have build a proof of concept test-bed (see Figure 1) which also we will make available to download in the future. The test-bed uses Minecraft ${ }^{1}$ as a virtual environment. The agents are controlled through behavior trees written in Python. Communication between the agents in Minecraft and their control software in Python is facilitated through the Spigot server ${ }^{2}$ and the pySpigot library ${ }^{3}$.

As an initial benchmark, we have implemented a modified scenario of open-ended foraging (Witkowski and Ikegami,

[^11]![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-209.jpg?height=424&width=840&top_left_y=269&top_left_x=1073)
Figure 1: Screenshot from the Foraging scenario in the testbed

2019). Agents are situated in an enclosed environment (see Figure 1), in which they can otherwise freely move. Additionally, agents can send and receive messages either as broadcasts or over different communication channels. Agents need to collect energy by staying close to certain blocks in the environment. If they have collected enough energy, they generate off-spring in the form of another agent in the environment. The off-spring agent is controlled by a mutated form of the parent behavior tree. If an agent does not have enough energy left, or if it has reached its maximum life span, it will be removed from the environment.

## Future work

Our results show that the agents in our proof-of-concept scenario are able to innovate to a certain degree before reaching an equilibrium state from which no further change in behavior is detected. In the next steps, we aim to enhance the test-bed to allow for the creation of dynamic environments. We believe that this is necessary to overcome the convergence and equilibrium and will lead to lifelong learning of the collective, continuously adapting to the changed circumstances. However, this will likely not lead to increasingly complex behaviors, but rather to everchanging behaviors of fixed complexity.

In order to verify this hypothesis, we will work on implementing measures of open-endedness and comparing the degree of open-endedness exhibited by the system. In particular, we will focus on the three pillars discussed earlier.

With these quantifying measures of open-endedness, we will then set out to develop open-ended algorithms that generated behaviors of increasing complexity. We will make use of the modular structure of behavior trees to keep an archive of prior behaviors, as well as combining them into larger structures, resulting in increasingly complex behaviors.

Lastly, we will then apply the developed framework and algorithms to a second type of embodied collectives: swarms of mobile robots. We will then investigate if our proposed methodology transfers across embodiments and if not, how we can ensure better transferability.

## Acknowledgements

We thank Carlos Gershenson-Garcia, Claire Glanois, and Petter Ögren for their insightful discussions and valuable feedback, which significantly shaped the direction of this project. JK acknowledges support from the Zukunftskolleg and from the Carl-Zeiss-Stiftung.

## References

Altera, Ahn, A., Becker, N., Carroll, S., Christie, N., Cortes, M., Demirci, A., Du, M., Li, F., Luo, S., et al. (2024). Project Sid: Many-agent simulations toward AI civilization. arXiv preprint arXiv:2411.00114.

Bonabeau, E., Dorigo, M., and Theraulaz, G. (1999). Swarm intelligence: from natural to artificial systems. Number 1. Oxford university press.

Brambilla, M., Ferrante, E., Birattari, M., and Dorigo, M. (2013). Swarm robotics: a review from the swarm engineering perspective. Swarm Intelligence, 7(1):1-41.

Brooks, R. A. (1991). Intelligence without representation. Artificial intelligence, 47(1-3):139-159.

Camazine, S., Franks, N. R., Sneyd, J., Bonabeau, E., Deneubourg, J.-L., and Theraula, G. (2001). Self-Organization in Biological Systems. Princeton University Press, USA.

Celemin, C., Pérez-Dattari, R., Chisari, E., Franzese, G., de Souza Rosa, L., Prakash, R., Ajanović, Z., Ferraz, M., Valada, A., and Kober, J. (2022). Interactive imitation learning in robotics: A survey. Foundations and Trends® in Robotics, 10(1-2):1-197.

Colledanchise, M. and Ögren, P. (2018). Behavior trees in robotics and AI: An introduction. CRC Press.

Dong, Y., Zhu, X., Pan, Z., Zhu, L., and Yang, Y. (2024). Villageragent: A graph-based multi-agent framework for coordinating complex task dependencies in minecraft. arXiv preprint arXiv:2406.05720.

Gaier, A., Asteroth, A., and Mouret, J.-B. (2019). Are quality diversity algorithms better at generating stepping stones than objective-based search? In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 115-116.

Grbic, D., Palm, R. B., Najarro, E., Glanois, C., and Risi, S. (2021). Evocraft: A new challenge for open-endedness. In International Conference on the Applications of Evolutionary Computation (Part of EvoStar), pages 325-340. Springer.

Hughes, E., Dennis, M., Parker-Holder, J., Behbahani, F., Mavalankar, A., Shi, Y., Schaul, T., and Rocktaschel, T. (2024). Open-endedness is essential for artificial superhuman intelligence. arXiv preprint arXiv:2406.04268.

Iovino, M., Scukins, E., Styrud, J., Ögren, P., and Smith, C. (2022). A survey of behavior trees in robotics and AI. Robotics and Autonomous Systems, 154:104096.

Kuckling, J. (2023). Recent trends in robot learning and evolution for swarm robotics. Frontiers in Robotics and AI, 10:1134841.

Lehman, J. and Stanley, K. O. (2011). Novelty search and the problem with objectives. In Genetic programming theory and practice IX, pages 37-56. Springer.

Pfeifer, R. and Iida, F. (2004). Embodied artificial intelligence: Trends and challenges. Lecture notes in computer science, pages 1-26.

Pugh, J. K., Soros, L. B., and Stanley, K. O. (2016). Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3:40.

Şahin, E. (2004). Swarm robotics: From sources of inspiration to domains of application. In International workshop on swarm robotics, pages 10-20. Springer.

Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019). Habitat: A platform for embodied AI research. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9339-9347.

Stanley, K. O. (2019). Why open-endedness matters. Artificial life, 25(3):232-235.

Stanley, K. O. and Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies. Evolutionary computation, 10(2):99-127.

Stepney, S. (2021). Modelling and measuring open-endedness. Artificial Life, 25(1):9.

Wang, Z., Cai, S., Liu, A., Jin, Y., Hou, J., Zhang, B., Lin, H., He, Z., Zheng, Z., Yang, Y., et al. (2024). Jarvis-1: Openworld multi-task agents with memory-augmented multimodal language models. IEEE Transactions on Pattern Analysis and Machine Intelligence.

Witkowski, O. and Ikegami, T. (2019). How to make swarms openended? evolving collective intelligence through a constricted exploration of adjacent possibles. Artificial life, 25(2):178197.

# Oscillatory Dynamics as an Universal Substrate for Computation: From Neural Circuits to Analog Electronics and Beyond 

Felix Effenberger ${ }^{1,2}$<br>${ }_{1}^{1}$ Ernst Strüngmann Institute, Frankfurt, Germany ${ }^{2}$ Natural Intelligence GmbH, Frankfurt, Germany felix @ nisys.ai


#### Abstract

Oscillatory dynamics are an ubiquitous feature of biological neural networks, yet their computational role has remained debated. Recent work has advanced the hypothesis that oscillations are not epiphenomenal but constitute a fundamental mechanism for information processing. Here, we synthesize three lines of research that address this question in conceptual, computational, and physical domains. First, theoretical considerations show that oscillations support functions based on synchrony, resonance and wave interference that are not accessible to non-oscillatory models. Second, the Harmonic Oscillator Recurrent Networks (HORN) model demonstrates that oscillatory dynamics confer advantages in learning speed, robustness, and parameter efficiency compared to non-oscillating recurrent architectures in the field of machine learning. Third, an analog-electronic implementation of the HORN model demonstrates the feasibility of exploiting transient oscillatory dynamics for computation in physical systems. This approach establishes that across neural, artificial, and physical systems, oscillatory transients provide a powerful resource for computation. Moreover, recent evidence from photonic, mechanical, and fluid systems shows that wave-based computation extends far beyond neural substrates. Together, these results show that transient oscillatory dynamics can serve as a unifying substrate for analog computation across a large variety of domains, offering new directions for energy-efficient, life-like information processing systems.


## Submission type: Late Breaking Abstracts

## Introduction

Artificial neural networks (ANNs) have achieved substantial success in diverse machine learning tasks, yet remain energy intensive and structurally distinct from biological neural circuits LeCun et al. (2015). In particular, conventional ANNs are still dominated by feed-forward architectures that depend on static activation states and synchronous updates. Cortical networks on the other hand are strongly recurrent Oldenburg et al. (2024) and their dynamics are dominated by asynchronous and ongoing information flow, as well as oscillatory activity spanning multiple frequency bands Buzsáki and Draguhn (2004).

Whether oscillations play a mechanistic role in computation or simply reflect epiphenomenal byproducts of neural activity has been a subject of longstanding discussions Sejnowski and Paulsen (2006); Singer (2018). New evidence from neuroscience suggests that oscillations facilitate information processing (Effenberger et al., 2024). Moreover, recent work demonstrates that oscillatory computation extends far beyond neural systems, with wave-based information processing emerging in photonic, mechanical, and fluid substrates (Hughes et al., 2019; Tanaka et al., 2019).

We hypothesize that the evolutionary convergence on oscillatory and wave-based dynamics across biological systems, from the molecular scale Casani-Galdon and GarciaOjalvo (2022); Xiong and Garfinkel (2023); Ford et al. (2023) to cortical dynamics Singer (2021); Heitmann et al. (2013), reflects deep computational advantages that transcend specific implementations Rapp (1987). Consequently, we propose that any life-like information processing system should exploit such dynamics to support its function. To examine oscillatory computation systematically, we combine three strands of research: Conceptual ideas of oscillatory dynamics, an in-silico simulation of oscillatory dynamics by the HORN model, and its physical implementation in analog-electronic hardware.

## Oscillatory Dynamics and the HORN model

Analytical work and simulation studies converge on the conclusion that oscillations provide distinct advantages for information processing (Todri-Sanial et al., 2024). Synchronization enables selective communication among subnetworks, resonance enhances frequency-specific responses, and fading memory allows temporal integration over multiple scales. These mechanisms permit robust transient computation without requiring convergence to fixed-point attractors Fakhoury et al. (2025).

In particular, the Harmonic Oscillator Recurrent Network (HORN, see Fig. 1) models each unit as a damped harmonic oscillator (Effenberger et al., 2024). Simulations of HORNs demonstrate improved parameter efficiency and learning speed relative to non-oscillatory recurrent net-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-213.jpg?height=371&width=846&top_left_y=264&top_left_x=149)
Figure 1: Harmonic Oscillator Recurrent Network (HORN) model as a recurrent network of coupled dampled harmonic oscillator (DHO) units with natural frequency $\omega$ and a damping factor $\gamma$, each representing the aggregate activity of a population of recurrently coupled excitatory (E) and inhibitory (I) biological neurons, processing temporally organized stimuli using transient oscillatory dynamics.

works, while exhibiting enhanced robustness under perturbations and noise.

The reason for the improved performance in HORNs results from several computational principles absent from conventional non-oscillating recurrent networks: dual coding through amplitude and phase enables simultaneous representation of features and their relationships; in-memory computing stores information in phase configurations while processing new inputs through interference; resonancebased computation achieves selectivity through frequencyspecific activation rather than mere labeled-coding by weight adjustment; high-dimensional phase space provides computational capacity exceeding physical degrees of freedom metastable dynamics maintain networks between order and chaos for continuous adaptation; and symbolic manipulation through phase relationships enables compositional representations Zheng et al. (2023).

Energy efficiency emerges as an unexpected advantage of oscillatory computation: while maintaining oscillations requires continuous energy input, computation occurs through passive wave interference, requiring no additional energy beyond sustaining the oscillations. This contrasts with digital computation where each operation consumes energy proportional to bit flips Bennett (1982).

To test the feasibility of implementing the aforementioned computational principles in physical substrates and overcome the conceptual slowness of simulating recurrent dynamics in digital von Neumann architectures, a small-scale HORN was implemented on analog hardware (Carvalho et al., 2025), showing that the analog implementation was capable of faithfully reproducing the dynamics of the digital twin.

Moreover, recent work shows that quantum-like computational properties can emerge naturally from classical oscillatory dynamics in HORNs through wave interference where multiple oscillatory signals combine through phase-
dependent addition to form superposition-like states (Khrennikov et al., 2025).

Importantly, wave-based and oscillatory computation is not limited to neural systems. Recent demonstrations across diverse physical substrates provide compelling evidence that wave dynamics constitutes a fundamental computational resource Macho-Ortiz et al. (2023); Perkins (2025) and wave physics itself can function as an analog recurrent neural network (Hughes et al., 2019). These diverse implementations demonstrate that resonance, interference, and synchrony can support machine learning tasks across physical platforms (Tanaka et al., 2019; Yu et al., 2024). The convergence of these approaches suggests that oscillatory dynamics represents a generalizable computational resource that transcends specific material substrates (Hughes et al., 2019; Tanaka et al., 2019).

## Discussion

Taken together, these findings support the view that oscillatory transients constitute a viable substrate for computation across biological, artificial, and physical systems (Singer and Effenberger, 2025; Effenberger et al., 2024; Carvalho et al., 2025). By integrating neuroscientific theory, computational modeling, and analog implementation, the work establishes a cross-domain foundation for wave-based information processing.

Several implications follow. First, oscillatory networks achieve parameter efficiency and robustness that may be advantageous in resource-constrained environments. Second, analog instantiations of oscillator networks highlight the potential for energy-efficient hardware that circumvents limitations of digital architectures. Third, the emergence of quantum-like properties from classical oscillatory dynamics bridges the gap between neural computation and quantum-like computations (Khrennikov et al., 2025), suggesting that wave interference provides a universal substrate for non-classical information processing. Fourth, the generality of oscillatory principles across diverse substrates, from photonic interference (Macho-Ortiz et al., 2023) to water wave dynamics (Maksymov, 2023) and mechanical oscillators (Perkins, 2025) - suggests a unifying framework for analog computation that extends beyond the neural paradigm. Fifth, a consciousness hypothesis emerging from this framework proposes that subjective experience arises from largescale phase coherence patterns across cortical networks, representing a phase transition where local oscillations become globally coordinated Bardon et al. (2025). This convergence indicates that wave-based computation represents a fundamental principle of information processing across physical systems (Hughes et al., 2019; Tanaka et al., 2019).

Implementing such principles in artificial systems can thereby be expected to open new pathways for the design of energy-efficient and adaptive information processing systems capable of life-like computation.

## References

Bardon, A. G., Ballesteros, J. J., Brincat, S. L., Roy, J. E., Mahnke, M. K., Ishizawa, Y., Brown, E. N., and Miller, E. K. (2025). Convergent effects of different anesthetics on changes in phase alignment of cortical oscillations. Cell Reports, 44(5):115685.

Bennett, C. H. (1982). The thermodynamics of computationa review. International Journal of Theoretical Physics, 21(12):905-940.

Buzsáki, G. and Draguhn, A. (2004). Neuronal Oscillations in Cortical Networks. Science, 304(5679):1926-1929.

Carvalho, P., Ulmann, B., Singer, W., and Effenberger, F. (2025). An analog-electronic implementation of a harmonic oscillator recurrent neural network. (arXiv:2509.04064).

Casani-Galdon, P. and Garcia-Ojalvo, J. (2022). Signaling oscillations: Molecular mechanisms and functional roles. Current Opinion in Cell Biology, 78:102130.

Effenberger, F., Carvalho, P., Dubinin, I., and Singer, W. (2024). The functional role of oscillatory dynamics in neocortical circuits: A computational perspective. Proceedings of the National Academy of Sciences of the United States of America, 122(4):e2412830122.

Fakhoury, T., Turner, E., Thorat, S., and Akrami, A. (2025). Models of attractor dynamics in the brain. (arXiv:2505.01098).

Ford, H. Z., Manhart, A., and Chubb, J. R. (2023). Controlling periodic long-range signalling to drive a morphogenetic transition. Elife, 12:e83796.

Heitmann, S., Boonstra, T., and Breakspear, M. (2013). A Dendritic Mechanism for Decoding Traveling Waves: Principles and Applications to Motor Cortex. PLoS Computational Biology, 9(10):e1003260.

Hughes, T. W., Williamson, I. A. D., Minkov, M., and Fan, S. (2019). Wave physics as an analog recurrent neural network. Science Advances, 5(12).

Khrennikov, A., Iriki, A., and Basieva, I. (2025). Constructing a bridge between functioning of oscillatory neuronal networks and quantum-like cognition along with quantuminspired computation and AI. (arXiv:2506.00040).

LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature, 521(7553):436-444.

Macho-Ortiz, A., Pérez-López, D., Azaña, J., and Capmany, J. (2023). Analog Programmable-Photonic Computation. Laser \& Photonics Reviews, 17(10).

Maksymov, I. S. (2023). Analogue and Physical Reservoir Computing Using Water Waves. (arXiv:2306.09095).

Oldenburg, I. A., Hendricks, W. D., Handy, G., Shamardani, K., Bounds, H. A., Doiron, B., and Adesnik, H. (2024). The logic of recurrent circuits in the primary visual cortex. Nature Neuroscience, 27(1):137-147.

Perkins, E. (2025). The mechanical duffing adaptive oscillator physical reservoir computer. Mechanical Systems and Signal Processing, 233:112711.

Rapp, P. E. (1987). Why are so many biological systems periodic? Progress in Neurobiology, 29(3):261-273.

Sejnowski, T. J. and Paulsen, O. (2006). Network Oscillations: Emerging Computational Principles. The Journal of Neuroscience, 26(6):1673-1676.

Singer, W. (2018). Neuronal oscillations: Unavoidable and useful? European Journal of Neuroscience, 48(7):2389-2398.

Singer, W. (2021). The cerebral cortex: A delay-coupled recurrent oscillator network? pages 3-28. Springer.

Singer, W. and Effenberger, F. (2025). Oscillations in Natural Neuronal Networks; An Epiphenomenon or a Fundamental Computational Mechanism? Human Arenas, pages 1-23.

Tanaka, G., Yamane, T., Héroux, J. B., Nakane, R., Kanazawa, N., Takeda, S., Numata, H., Nakano, D., and Hirose, A. (2019). Recent advances in physical reservoir computing: A review. Neural Networks, 115:100-123.

Todri-Sanial, A., Delacour, C., Abernot, M., and Sabo, F. (2024). Computing with oscillators from theoretical underpinnings to applications and demonstrators. npj Unconventional Computing, 1(1).

Xiong, L. and Garfinkel, A. (2023). Are physiological oscillations 'physiological'? (arXiv:2301.08996).

Yu, W., Guo, H., Xiao, J., and Shen, J. (2024). Physical neural networks with self-learning capabilities. Science China Physics, Mechanics \& Astronomy, 67(8):287501.

Zheng, H., Lin, H., and Zhao, R. (2023). Representing part-whole hierarchy with coordinated synchrony in neural networks. https://openreview.net/forum?id=JAfGlmRBTU.

# Quantifying Microslips in Tele-operarion Task 

Takaya Mori ${ }^{1}$, Yuki Tanaka ${ }^{1}$, Seiichiro Katsura ${ }^{1}$, and Eiko Matsuda ${ }^{1}$,<br>¹Department of System Design Engineering, Keio University, Japan<br>tmori@katsura.sd.keio.ac.jp


#### Abstract

Human actions, although seemingly simple, often consist of small corrective movements known as microslips. These have been qualitatively observed in daily activities, such as making coffee, and categorized into types including hesitations, trajectory shifts, action stutters, and hand shape changes. Previous studies have suggested that microslips emerge inevitably from bottom-up, life-like action selection processes, but quantitative analyses have been lacking. In this study, we conducted a pilot robot tele-operation experiment to record and quantify microslips during a block-stacking task. We successfully quantified the microslips observed during the experiment by analyzing three-dimensional trajectory plots of the robot's end-effector.


## Submission type: Late Breaking Abstracts

## Introduction

The actions we routinely perform in our daily lives may appear simple at first glance, but in fact they consist of a continuous sequence of small transitions that respond to various objects and situations. When one observes in detail the movements of the hand manipulating objects, these transitions are not always smooth; rather, moments frequently arise in which the flow of action appears to stagnate. For example, in the task of making instant coffee, we often correct our actions, such as reaching toward the cup but then redirecting the trajectory toward the spoon to scoop the coffee powder. In this way, even within tasks that appear simple, the hand's direction may suddenly change, lightly touch an object, or abruptly stop its motion. Reed et al. carefully observed the making of instant coffee with materials placed on a table, and named such small corrective actions "microslips," categorizing them into four types: "hesitations," "trajectory shifts," "action stutters," and "hand shape changes(Reed et al., 2009)." In the context of artificial life research, interpreted microslips as parallel cognitive processes in human subconsciousness, and proposed a simple computational model of microslips(Ogai and Ikegami, 2008). In their experiment, simulated agents performing a task of reaching either of two objects exhibited sudden
switches from a trajectory toward one object to a trajectory toward the other object. This behavior was interpreted as an implementation of "trajectory shifts" in microslips by means of a mathematical model. The final choice of which object to reach was shown to possess a riddled basin structure, making it dynamically undecidable. In other words, microslips are not pre-designed, but rather inherently embedded in the system and inevitably manifested.

Robots, on the other hand, are primarily top-down, in which a global "action plan" is determined in advance and local actions are executed accordingly. Then, what is the advantage of bottom-up action planning characteristic of life? What benefit might a system that inherently includes microslips possess? These questions can be addressed by constructing a mathematical model of action selection that generates microslips, and by clarifying the dynamical properties of such systems (e.g., stability and boundary structures). Previous studies on microslips were based on qualitative observations by experimenters, and quantitative analyses have not been conducted. Therefore, in this study, we precisely recorded and analyzed microslips occurring while humans perform global tasks, using tele-operation tasks. Participants were given a task to operate the robot to build a complex block structure. During the process of solving this global block-stacking task, participants inevitably faced local switches of action. We then analyzed the trajectories of the trials in which microslips occurred and attempted their quantitative detection.

## Methods and Results

The experimental environment is shown in Figure 1. The participants were instructed to remotely operate the robot to stack the symmetrical objects by placing the larger ones at the bottom and then stacking the smaller ones on top in order. The participants controlled the leader robot located at the left side in Figure 1 (A). The follower robot tracked the position of the leader robot, and the participants performed the block-stacking task while directly observing the positional relationship between the follower robot and the blocks. Two participants took part as volunteers, and each

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-217.jpg?height=945&width=855&top_left_y=264&top_left_x=146)
Figure 1: (A) shows an overview of the experimental setup. 1: Workspace. 2, 3, 4: Blocks to be stacked by the participant. Block 2, the largest, serves as the base, with Blocks 3 and 4 stacked in order of decreasing size. 5: Dummy block. (B) shows the actual experimental process in chronological order. The participant begins operating the robot from the initial state in (1). In (2), the participant attempts to lift the wrong block, causing an action stutter, but then corrects the trajectory, grasps the correct block in (3), and stacks it in (4).

was asked to perform the block-stacking task ten times. Before each trial, the experimenter arranged the blocks: for one participant, the block arrangement was fixed across all trials, while for the other participant, the arrangement was randomized each time.

The participant with blocks arranged randomly each time exhibited action stutters during the ninth trial, as shown in Figure 1 (B) (2). After realizing the mistake, the participant continued the block-stacking task in the correct order. The three-dimensional trajectory plot of the end-effector during this sequence is shown in Figure 2.In Figure 2, the trajectory is visualized with the starting point shown as a blue circle, the endpoint as a red circle, and the flow along the time series represented by a gradient. Within this trajectory, the action stutter corresponding to Figure 1(2) is indicated by a green circle. The definition of action stutters consists of two conditions: 1. the current trajectory overlaps with a small neighborhood (within 0.005 m ) of a trajectory that was previously traversed, with the revisit occurring at least 5

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-217.jpg?height=894&width=885&top_left_y=278&top_left_x=1080)
Figure 2: 3D plot of the end-effector trajectory during blockstacking trials in which action stutters occurred.The trajectory corresponds to Figure 1: it starts at (1), then an action stutter occurs at (2), followed by a trajectory correction toward (4). After that, the participant continued stacking the blocks in the correct order.

seconds after the initial traversal; and 2 . this neighborhood (with a radius of 0.01 m ) does not correspond to either the initial position or the block-stacking position.

## Discussion, Conclusion and Future Works

In this study, we conducted a pilot experiment of a robot tele-operation task with a small number of participants and confirmed that microslips occurred more frequently in the more complex random block arrangement task. This finding is consistent with previous studies (Reed, 2009). Furthermore, we successfully detected microslips in trajecto-ries-something that had never been attempted before.In future work, we plan to use the collected data to analyze all four categories of microslips and construct a model that reflects human action dynamics, and we aim to clarify the adaptive properties of systems that inherently include microslips from a dynamical perspective. The definition and detection of microslips established in this study make a significant contribution to the mathematical modeling of bottom-up, life-like human behavior, and we will continue to refine and formalize this approach.

## References

Ogai, Y. and Ikegami, T. (2008). Microslip as a simulated artificial mind. Adaptive Behavior, 16(2-3):129-147.

Reed, E. S., Palmer, C. F., and Schoenherr, D. (2009). On the nature and significance of microslips in everyday activities. Japanese Journal of Ecological Psychology, 4(1):51-66.

# Using Human Feedback and Reward Modeling to Search for Artificial Life 

Eugène Bergeron ${ }^{1}$ and Vassilis Papadopoulos ${ }^{1 *}$<br>${ }^{1}$ CSFT, EPFL, Switzerland;


#### Abstract

We introduce AlifeHFPipeline, an end-to-end pipeline that makes it easy to collect human evaluations about Artificial Life dynamics, and use them to train a reward model. The reward model can then be used as a proxy for human preference, allowing large-scale search of Artificial Life parameters to discover interesting behaviours. We describe the AlifeHFPipeline, and test it on 2D binary CAs, Lenia and Particle Life. The results confirm the proof-of-concept and show that this approach is a promising direction for automatic parameter search. We finish by discussing current limitations and ongoing work to overcome them.


Submission:Late Breaking Abstract. Code [at this link]. Videos [at this link]

## Introduction

When designing new artificial worlds, one often ends up with a promising set of 'rules', or 'physics', that usually depend on a set of free parameters. For instance, NCAs (Mordvintsev et al., 2020) depend on 8000 neural network parameters, Lenia (Chan, 2020) depends on means and variances of a set of Gaussian bumps and 2-dimensional binary CAs(Kari, 2005) depend on the choice of update rule among $>10^{150}$ possibilities. One also needs to choose how to sample initial conditions for the world, as the dynamics may vary greatly with different initial conditions(Behrens et al., 2024). Of this sea of parameters, typically only a small fraction will produce interesting results (Papadopoulos et al., 2024; Chan, 2019), and one has to navigate it to discover what the artificial world is capable of. The main problem is that this process is hard to automate, chiefly because the notion of 'interestingness' of dynamics is broad, fuzzy, and generally hard to define.

It is, in contrast, generally easy for a human to decide whether some Alife dynamics look interesting, even with little to no experience with the specific model being simulated. Using human judgment is often how one selects model parameters, sometimes combined with genetic algorithms by selecting and mutating promising candidates (Slackermanz, 2023; Christiano et al., 2017; Takagi, 2001). While these approaches are effective, they are inherently unscalable due to the human-in-the-loop. Reinforcement Learning with Human Feedback (RLHF)(Christiano et al., 2017) was introduced to solve the scalability problem, while still benefiting from human intuition. In a nutshell, it uses a 'reward model', a neural network trained to mimic human preference from a

[^12]modest amount of examples. Once trained, it can be used as a proxy for human judgment, allowing the use of human intuition at scale.

The goal of this work is to apply reward modeling for the specific task of assigning an 'interestingness' score to Alife dynamics. To this end, we introduce [AlifeHFPipeline], an end-to-end, model-agnostic pipeline, which makes it easy to collect human evaluations and use them to train and test the reward model. In this short note, we present the pipeline and its various features. Then, we display encouraging preliminary results, and finally discuss the current limitations and areas that still require work.

## AlifeHFPipeline

The main goal of the HF pipeline is to train a 'Rewarder' model to assign scores to Alife dynamics in such a way that a high score corresponds to a high preference from humans ${ }^{1}$.

The pipeline is a loop consisting of 4 stages. Generation A Generator samples parameters for an Alife system. Simulation A Simulator simulates the system for each of the sampled parameters. Labeling A human is presented with sets of 4 simulations, and is tasked to rank them according to their preference. Training A Rewarder is trained on the labeled data, learning a score which attempts to reproduce the human rankings. Generator Tuning Using the Rewarder, one can optionally train the Generator to generate higher score parameters on average.

This loop can be repeated, this being especially useful if the Generator is tuned, as subsequent loops will produce higher-scoring candidates, helping refine the Rewarder by providing more meaningful labels.

Our codebase provides a scaffold (CLI and GUI) that allows implementation of the Generator, Simulator, and Rewarder. One needs only define how to sample the parameters, how to run the simulation given a set of parameters, and define the Neural Network that will be used as the Rewarder (defaults are provided).

## Preliminary results

To test the soundness of the method, we tested the pipeline using Lenia, Particle Life, and 2D CAs as substrates. In all cases, the preference metric we aimed for was 'interestingness'. Generally, we looked for behaviours that were surprising, unexpected, and more broadly, 'not boring'. The

[^13]advantage of providing feedback by ranking videos is that it is easy to tell when a video is 'more interesting' than another.

For each of the three models, we rank 100 sets of videos, which results in 600 labeled pairs. The loss is binary crossentropy, where output probabilities are computed as in ELO from the scores assigned by the rewarder, see (1) :

$$
\begin{aligned}
& \operatorname{loss}=-h f(1) \log P\left[v_{1} \succ v_{2}\right]-h f(2) \log P\left[v_{2} \succ v_{1}\right] \\
& P\left[v_{1} \succ v_{2}\right]=\frac{e^{r_{1}}}{e^{r_{1}}+e^{r_{2}}}, P\left[v_{2} \succ v_{1}\right]=\frac{e^{r_{2}}}{e^{r_{1}}+e^{r_{2}}}
\end{aligned}
$$

Where $v_{i}$ are the videos of dynamics, $r_{i}$ the scores assigned by the rewarder, and $h f(i)$ is human feedback, equal to 1 if $i$ is preferred, 0.5 if neither is preferred, 0 otherwise.

For our experiment, we kept the Generator simple and didn't experiment with trainable Generators. For 2D CA, it generates a random outer-totalistic rule. For ParticleLife, it chooses a random number of particle species, and generates a matrix of interaction (repulsive, attractive, neutral). For Lenia, we generate parameters using the 'smart' prior given in (Papadopoulos et al., 2024), which tends to create non-trivial behaviours more often than uniform sampling. In all cases, we generate 10 -second videos at 30 fps for human labeling. Initial conditions in all cases are appropriate noise.

Finally, the most important choice is the model used as the Rewarder. Indeed, the main constraint is that it needs to be trainable with a relatively small dataset i.e., less than a few thousand datapoints. In our case, the input to the model is the full video of the dynamics, given that is what the human is judging ${ }^{2}$ This makes it very hard to train a model from scratch while preventing overfitting, since the model would first need to learn good features for videos, and then use those features to assign the preference scores. The solution here is to leverage pretrained models that already contain good general features, and finetune them on the preference dataset, something that requires much fewer examples to obtain good results(Finn et al., 2017).

From some testing, we find that currently the best performing model is Clip-VIP (Xue et al., 2023), which is a video embedding model, itself finetuned on videos from a CLIP(Radford et al., 2021) baseline. For finetuning, we find that the more efficient approach is to freeze the model's weights and train a small transformer layer on top of ClipVIP's features, to output the reward. Note that ClipVIP accepts only up to 16 frames of video, so we select 16 equally spaced frames from the videos.

Table 1 summarizes the performance of the Rewarder after being trained for 10 epochs. Note that accuracy is computed only for non-tied pairs. We see that the loss is far from

[^14]|  | CA2D | Lenia | ParticleLife |
| :--- | :---: | :---: | :---: |
| Validation Accuracy | $87 \%$ | $91 \%$ | $86 \%$ |
| Validation Loss | 0.43 | 0.42 | 0.37 |
| Minimal possible loss | 0.16 | 0.24 | 0.06 |

Table 1: Metrics for the Rewarder trained on 600 preference pairs. The minimal loss is non-zero due to tied pairs.

optimal, though accuracy is high. To get a better feeling for the quality of the Rewarder beyond metrics, we generate batches of 30 parameters, then keep only the $20 \%$ best scoring, to compose a dataset of filtered rule, which can be viewed at [this link]. Generally, we find that the rewarder is very good at filtering the bulk of the data, but is still lacking in differentiating between two high-scoring rules, which is not surprising given the data contains very few of those examples, due to their rarity.

## Limitations and further work

We have shown that, as a proof of concept, reward modeling with human feedback is a promising direction to automate the search of parameter space in Alife systems. Further work is still needed to better evaluate the robustness and usefulness of the reward models. The first improvement we are working on is to involve the rewarder into the labeling process, to better select which pairs are sent to the human to maximize their usefulness. This will particularly help in cases (such as CA2D and Lenia in our experiments) where the initial Generator generates mostly uninteresting parameters, resulting in low-signal human evaluations. Another direction we are exploring is using the reward model with evolutionary algorithms to generate parameters that maximize 'interestingness'. Not only is this a good way to generate parameters at scale, but it is also a good probe for weaknesses in the learned reward, as the evolutionary algorithm may 'reward-hack'. Hopefully, such parameter-generating methods can also be used as an improved Generator, providing more interesting videos for human feedback, making the whole pipeline more data-efficient.

Currently, the main limitation is the lack of good pretrained video models. Indeed, we haven't been able to find a foundational model for video features, and most models we found were limited to very few frames (like CLIP-Vip), or trained only on a subset of videos (e.g., HowTo(Miech et al., 2019)) which would not generalize well to Alife videos. A similar problem was faced in other approaches leveraging pretrained model(Kumar et al., 2025; Khajehabdollahi et al., 2025), where often researchers opt to use CLIP on freezeframes instead. Since the field of AI moves very fast, we hope some better options appear in the near future. Another option would be to collect a dataset of Alife videos, and train an 'Alife' foundation model, which could be used in this application and many others!

## Acknowledgements

We would like to thank Clément Hongler, Barbora Hudcová, Slackermanz, Ehsan Pajouheshgar, Arthur Renard, Guilhem Doat, Paul Weng for helpful discussions regarding this work. V.P. would also like to thank Pauline Lamotte, Ioannis Papadopoulos, and Armelle Anthoine for help with labeling in the early stages of this work. This work was partially supported by an EPFL FSB Seed Funding Grant.

## References

Behrens, F., Hudcová, B., and Zdeborová, L. (2024). Dynamical phase transitions in graph cellular automata. Phys. Rev. E, 109:044312.

Chan, B. W.-C. (2019). Lenia: Biology of artificial life. Complex Systems, 28(3):251-286.

Chan, B. W.-C. (2020). Lenia and Expanded Universe. The 2020 Conference on Artificial Life, pages 221-229.

Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., and Amodei, D. (2017). Deep reinforcement learning from human preferences.

Finn, C., Abbeel, P., and Levine, S. (2017). Model-agnostic metalearning for fast adaptation of deep networks.

Kari, J. (2005). Theory of cellular automata: A survey. Theoretical computer science, 334(1-3):3-33.

Khajehabdollahi, S., Hamon, G., Cvjetko, M., Oudeyer, P.-Y., Moulin-Frier, C., and Colas, C. (2025). Expedition \& expansion: Leveraging semantic representations for goal-directed exploration in continuous cellular automata.

Kumar, A., Lu, C., Kirsch, L., Tang, Y., Stanley, K. O., Isola, P., and Ha, D. (2025). Automating the search for artificial life with foundation models.

Miech, A., Zhukov, D., Alayrac, J., Tapaswi, M., Laptev, I., and Sivic, J. (2019). Howto100m: Learning a text-video embedding by watching hundred million narrated video clips. $C o R R$, abs/1906.03327.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing Neural Cellular Automata. Distill, 5(2):e23.

Papadopoulos, V., Doat, G., Renard, A., and Hongler, C. (2024). Looking for complexity at phase boundaries in continuous cellular automata.

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. (2021). Learning transferable visual models from natural language supervision.

Slackermanz (2023). MNCA Compilation.
Takagi, H. (2001). Interactive evolutionary computation: fusion of the capabilities of ec optimization and human evaluation. Proceedings of the IEEE, 89(9):1275-1296.

Xue, H., Sun, Y., Liu, B., Fu, J., Song, R., Li, H., and Luo, J. (2023). Clip-vip: Adapting pre-trained image-text model to video-language representation alignment.

# Functional Information Decomposition: A Principled Approach to Information Decomposition 

Clifford Bohm ${ }^{1}$<br>${ }^{1}$ BEACON Center for the Study of Evolution in Action, Michigan State University. cliff.bohm@gmail.com


#### Abstract

Functional Information Decomposition (FID) is a novel approach to information decomposition. Decomposing information is critical to understanding complex systems and may be central to defining emergence. An effective decomposition assesses how each input (or part) contributes to an output (or whole), both independently and in combination. Prior methods, such as Partial Information Decomposition (PID), rely on observed data, which is often incomplete or biased. Such data can reveal statistical relationships but not structural dependencies. FID addresses this problem with a two-part solution: it defines exact decompositions when the function is fully specified, and uses sampling to compute bounded estimates when it is not. This yields interpretable results without overstating certainty.


## Submission type: Late Breaking Abstracts

## Introduction

Emergent phenomena, where wholes appear to exhibit properties greater than the sum of their parts, have been repeatedly observed in artificial life research (Bedau, 1997; Langton, 1989; Holland, 1998; Ray, 1991; Beer, 1995; Banzhaf, 1997). To understand and predict emergent behavior, we need rigorous methods to distinguish when collective properties truly exceed individual contributions. Information theory provides a natural framework to assess relational properties like these (Shannon, 1948). However, to date, there is no agreed-upon information-theoretic definition for emergence. One of the barriers to such a definition is that we also lack a formal method for decomposing mutual information. Such a decomposition would detail how individual parts of a system contribute to the behaviors and properties of the whole, and to what extent this contribution is attributed to properties of the parts versus properties that only emerge irreducibly when the parts interact.

The goal of decomposing mutual information is almost as old as the notion of mutual information itself. The issue was first considered via "interaction information" (McGill, 1954) and "total correlation measures" (Watanabe, 1960)
in the 1950s and 1960s, and more recently via Partial Information Decomposition (PID). However, after decades of effort, there remains no generally accepted framework for characterizing multivariate information structure (Kolchinsky, 2022; Gutknecht et al., 2023; Varley et al., 2024).

Among the concepts that information decomposition seeks to isolate, synergy is perhaps the most fundamental. Synergy captures the essence of emergent behavior: information that exists only when variables are jointly considered and is inaccessible from any individual variable alone. The XOR function is a canonical example of synergy, where the output cannot be predicted to any degree from either input individually but is fully determined when both inputs are known.

Although it may seem counterintuitive, measuring synergy becomes problematic when considering incomplete data. Suppose we observe only three of the four possible input-output pairs from an unknown deterministic Boolean function, and based on these observations alone, we determine that the underlying function is either OR or XOR. How can we assess synergy based on the partial data when these functions represent fundamentally different types of information processing, each with a unique degree of synergy?

This reveals an unavoidable constraint on synergy measurement. With only partial data, we cannot assess synergy directly because we cannot know the complete functional relationship that determines how inputs combine to produce outputs. This implies that synergy is a property of a function, not of data. Regardless of the data collected, if the underlying function is XOR, then the synergy is 1.0 bit. However, that same function is consistent with OR, which does not have 1.0 bits of synergy. Any valid approach must acknowledge that our partial data is inherently consistent with more than one possible synergy value - from the pure synergy of XOR to the mixed profile of OR.

The problem with current approaches is not that they give wrong answers, but that they give single answers when the data support a range of possibilities. A principled approach must either work with complete functional specifications or explicitly characterize the uncertainty about synergy that re-
mains when such specifications are unavailable.
We begin with a fundamental insight: a complete decomposition of the information relationships within a system requires full knowledge of the system's input-output mapping. Partial data may reveal statistical patterns in observed behavior and suggest likely causal pathways (as in Bohm et al. (2022)). However, unobserved conditions leave the underlying function underconstrained. As a result, partial data cannot support definitive system-wide claims. Any attempt to generalize beyond the observed data risks drawing conclusions that exceed what the data can justify.

Functional Information Decomposition (FID) is a novel solution to the problem of mutual information decomposition, offering a principled, function-based approach. When a complete specification is provided, FID leverages the fact that this ensures independence among the input variables; thus, the independent information provided by each input is simply the mutual information between that input and the output. It follows that synergy is the residual information that the inputs provide about the output that is not independently provided by any input.

In practice, complete functional data is often unavailable. In such cases, an exact decomposition is not possible. However, sampling the space of possible functions is a viable approach to establishing bounds on decompositions compatible with incomplete information. This approach enables meaningful inference while respecting the fundamental limitations imposed by partial data.

## Formal Overview

Let $X=\left\{X_{1}, X_{2}, \ldots, X_{n}\right\}$ denote a set of discrete, statistically independent input variables, and let $Y$ be a discrete output variable. Functional Information Decomposition (FID) is defined with respect to a complete function $f: X \rightarrow Y$ (for every possible input combination the output is defined), which may be deterministic or probabilistic.

- Total Information - The mutual information between all inputs and the output:

$$
I_{\mathrm{tot}}=I(Y ; X)
$$

- Independent Information for variable $X_{i}$ - The information about $Y$ that is provided by $X_{i}$ alone:

$$
I_{\mathrm{ind}}\left(X_{i}\right)=I\left(Y ; X_{i}\right)
$$

- Synergistic Information - The portion of $I(Y ; X)$ that is not attributable to any individual input, but arises only from their joint contribution:

$$
I_{\mathrm{syn}}(X \rightarrow Y)=I_{\mathrm{tot}}-\sum_{i=1}^{n} I_{\mathrm{ind}}\left(X_{i}\right)
$$

- Solo-Synergy for variable $X_{i}$ - The portion of synergy that is lost when $X_{i}$ is excluded, or how much of the total synergy depends specifically on $X_{i}$ 's participation:
$I_{\text {solo-syn }}\left(X_{i}\right)=I_{\text {syn }}(X \rightarrow Y)-I_{\text {syn }}\left(X \backslash X_{i} \rightarrow Y\right)$
(where $X \backslash X_{i}$ denotes the set of all inputs excluding $X_{i}$.)
- Information Loss for variable $X_{i}$ - The total loss in predictive power about $Y$ when $X_{i}$ is excluded from the set of inputs:

$$
I_{\text {loss }}\left(X_{i}\right)=I_{\text {ind }}\left(X_{i}\right)+I_{\text {solo-syn }}\left(X_{i}\right)
$$

Finally, all FID terms are non-negative.

## OR/XOR Example Illustrated

Consider $f\left(X_{1}, X_{2}\right) \rightarrow Y$ :

$$
\begin{aligned}
& f(0,0)=0 \\
& f(0,1)=1 \\
& f(1,0)=1 \\
& f(1,1)=\text { unknown }
\end{aligned}
$$

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-225.jpg?height=295&width=723&top_left_y=982&top_left_x=1135)
Figure 1: FID applied to the incomplete OR/XOR function. The black dot and black circle represent the deterministic completions, OR and XOR, respectively. Red points represent sampled probabilistic completions. The dashed red line indicates the maximum-entropy, "neutral" completion.

Figure 1 illustrates the decomposition of the incomplete OR/XOR function, showing that the possible solutions, in addition to the deterministic solutions for OR and XOR, include probabilistic solutions that collectively illustrate the space of all potential solutions (i.e., models) allowed under the data.

## Conclusion

Functional Information Decomposition provides a novel approach to information decomposition, yielding precise and human-interpretable results without overstating certainty. Critically, FID acknowledges that partial data can only offer partial constraints and represents this by maintaining a cloud of candidate solutions when data is insufficient to indicate a single solution.

FID should afford practitioners a more accurate assessment of functional relationships. Moreover, FID, and in particular its technical solution for synergy, could be a critical step in efforts to establish a more scientifically grounded definition for Emergence. FID is particularly well-suited for artificial life research, where agents and environments often exhibit underdetermined or partially specified dynamics, making statistical decompositions unreliable.

## Acknowledgments

Large Language Models (ChatGPT, Claude, and DeepSeek) were used to refine, but not create, content in this work.

## References

Banzhaf, W. (1997). Emergence and complexity in genetic programming. In Genetic Programming 1997: Proceedings of the Second Annual Conference, pages 69-78. Morgan Kaufmann.

Bedau, M. A. (1997). Weak emergence. Philosophical perspectives, 11:375-399.

Beer, R. D. (1995). A dynamical systems perspective on agentenvironment interaction. Artificial Intelligence, 72(1-2):173215.

Bohm, C., Kirkpatrick, D., Cao, V., and Adami, C. (2022). Information fragmentation, encryption and information flow in complex biological networks. Entropy, 24(5):735.

Gutknecht, H., Makkeh, A., and Wibral, M. (2023). From babel to boole: The logical organization of information decompositions. Entropy, 25(6):932.

Holland, J. H. (1998). Emergence: From Chaos to Order. Oxford University Press, Oxford, UK.

Kolchinsky, A. (2022). A novel approach to the partial information decomposition. Entropy, 24(2):154.

Langton, C. G. (1989). Artificial life. In Langton, C. G., editor, Artificial Life, volume 6, pages 1-47. Addison-Wesley.

McGill, W. (1954). Multivariate information transmission. Transactions of the IRE Professional Group on Information Theory, 4(4):93-111.

Ray, T. S. (1991). An approach to the synthesis of life. In Langton, C. G., Taylor, C., Farmer, J. D., and Rasmussen, S., editors, Artificial Life II, volume X, pages 371-408. Addison-Wesley.

Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3):379-423.

Varley, T. F. et al. (2024). A scalable, synergy-first backbone decomposition of higher-order structures in complex systems. npj Complex Systems, 1(1).

Watanabe, S. (1960). Information theoretical analysis of multivariate correlation. IBM Journal of research and development, 4(1):66-82.

# QD-LEAR: Exploring Quality-Diversity Tradeoffs in LLM-Evolved Agent Rule Representations 

Can Gurkan ${ }^{1}$, Narasimha K. Jwalapuram ${ }^{1}$, Kevin Wang ${ }^{1}$, Rudy Danda ${ }^{1}$, John Chen ${ }^{1}$, Leif Rasmussen ${ }^{1}$, Uri Wilensky ${ }^{1}$<br>${ }^{1}$ Northwestern University, Evanston, IL<br>gurkan@u.northwestern.edu


#### Abstract

Quality-Diversity (QD) algorithms have been widely applied across diverse domains by encouraging exploration and diversity, yet their impact on the representational quality of evolved solutions has garnered little attention to date. We address this gap using a framework that employs Large Language Model (LLM)-driven mutation to evolve agent controllers in NetLogo multi-agent simulations. By systematically varying the emphasis between fitness and novelty through a weighted sum method, we investigate how selection pressures influence the syntactic and semantic characteristics of evolved code. Experiments on a new multi-agent benchmark reveal that, while final fitness remains largely unaffected, greater emphasis on novelty consistently produces a higher proportion of qualitatively superior code structures, leading to more robust representations. These findings underscore a hidden cost of objective-centric optimization and demonstrate that diversity-driven search can enhance the representational quality of evolved code.


## Submission type: Late Breaking Abstract

Data/Code available at: https://github.com/ can-gurkan/LEAR/tree/qd-lear

## Introduction

Objective-based search is a dominant paradigm in artificial intelligence, where progress toward explicit objectives is rewarded (Lehman and Stanley, 2011; Kumar et al., 2025). Quality-Diversity (QD) algorithms offer an alternative paradigm by combining objective optimization with novelty pressure, enabling exploration of diverse behaviors and improving performance across domains (Colas et al., 2020; Gaier et al., 2020; Cully and Demiris, 2018; Tjanaka et al., 2022; Conti et al., 2018). Yet, an important question remains largely unexplored: how does the balance between quality (objective-based) and diversity (noveltybased) search influence the representational characteristics of the discovered solutions? In this study, we analyze how varying the fitness-novelty tradeoff shapes both syntax and semantics of agent controllers in multi-agent systems by using LLM-driven mutation to evolve interpretable NetLogo (Wilensky, 1999) code.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-228.jpg?height=389&width=671&top_left_y=831&top_left_x=1155)
Figure 1: Overview of the QD-LEAR framework.

## Related Work

Relatively few studies have examined how objective and diversity-based optimization strategies affect the representational structure of evolved solutions. Lehman and Stanley (2010) showed that novelty search outperformed objectivedriven algorithms, producing more efficient genetic programs. Woolley and Stanley (2011) found that when openendedly evolved images were re-optimized as explicit objectives, the resulting encodings became bloated and less structured. More recently, Kumar et al. (2025) showed that training neural networks by following objective gradients can yield functionally accurate but fractured-entangled representations, even when external behaviors appear similar. Together, these findings suggest that objective-based optimization may compromise representational quality, motivating our study of this question in the context of LLM-evolved code representations.

## Methods

Our approach extends the LEAR framework (Gurkan et al., 2025), where LLMs generate NetLogo rules for agents in a multi-agent grid world. Agents begin with a baseline exploratory rule and evolve through evaluation, QD-based selection, LLM-driven mutation, and replacement (Figure 1). Selection employs a weighted sum of fitness and novelty, parameterized by $\alpha$, ranging from purely objective-driven $(\alpha=0)$ to purely novelty-driven ( $\alpha=1$ ). Novelty is measured by Euclidean distance in a behavior space characterized by measuring the agent's current energy at evenly

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-229.jpg?height=346&width=844&top_left_y=260&top_left_x=150)
Figure 2: Depictions of the multi-agent NetLogo environment with different landscape functions, where blue and purple cells depict positive and negative energy values.

spaced time points. Once selected, agents undergo LLMpowered mutation. Agent code is provided to the LLM with prompts describing the environment, objectives, and constraints, enabling the generation of semantically meaningful variants. The mutated code is then evaluated within the simulation environment according to the quality-diversity (QD) criteria described above.

Environments We introduce a NetLogo multi-agent environment designed to foster creative strategies without constraining agent actions. Agents move on a $60 \times 60$ toroidal grid where each cell has a positive or negative energy value, accumulating fitness by maximizing energy intake while avoiding losses. Each agent can percieve its environment in various distances and directions, encouraging anticipatory strategies based on limited information. To generate diverse landscapes with varying complexity, we use the Rastrigin, Griewank, and Drop-Wave functions, which define cell energies and enable rigorous evaluation of adaptability and representational capacity (Figure 2).

## Experiments

We vary the trade-off parameter $\alpha$ across five values, running 10 trials per landscape function, and track three metrics: the fitness of the best agent, program length, and the proportion of semantically correct representations. To assess representational quality, we define five criteria tailored to our environment: use of all input variables, conditional energy checks, correct step sizes, aligned turning angles, and mechanisms for escaping local minima or exploiting high-energy regions. These criteria capture both functional correctness and alignment with the agent's sensory-action model.

## Results

Emphasizing novelty consistently improved representational quality across all landscapes. As Figure 3 shows, increasing the novelty weighting parameter $\alpha$ led to a higher proportion of correct representations, with agents more effectively integrating sensory inputs, using context-aware movement, and developing adaptive jump mechanisms. Importantly, this improvement did not reduce task performance, as fit-
ness scores remained statistically comparable across $\alpha$ values, and program length was unaffected.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-229.jpg?height=623&width=816&top_left_y=408&top_left_x=1087)
Figure 3: Mean proportion of correct code representations as a function of $\alpha$ across the three landscape functions.

## Limitations

While our study offers insights into how quality versus diversity-driven search affects the representational quality of LLM-evolved agent rules, several limitations remain. Our environment and evaluation criteria are domain-specific and may not generalize to other languages or more complex settings. The five representational quality checks, though interpretable, may overlook innovative behaviors and can misclassify valid or invalid representations due to the difficulty of assessing semantics. Moreover, agents were evolved independently, without accounting for collective dynamics, limiting applicability to multi-agent coordination. Future work should broaden domains, adopt more adaptive evaluation metrics, and explore co-evolutionary approaches.

## Conclusion

This study examined how the quality-diversity tradeoff influences the representational quality of LLM-evolved agent code. By varying the emphasis between fitness and novelty in a NetLogo environment and evaluating code against five domain-specific criteria, we found that greater emphasis on novelty produced more coherent and context-aware representations without compromising task performance, while objective-driven search often generated brittle code despite achieving comparable fitness. These findings underscore the hidden costs of objective-centric optimization and suggest that diversity-driven search, when combined with LLMpowered mutation, offers a promising avenue for evolving more interpretable and generalizable agent behaviors.

## Acknowledgements

This work was supported by NSF Grant POSE: Phase II.

## References

Colas, C., Madhavan, V., Huizinga, J., and Clune, J. (2020). Scaling map-elites to deep neuroevolution. GECCO 2020 - Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pages 67-75.

Conti, E., Madhavan, V., Such, F. P., Lehman, J., Stanley, K. O., and Clune, J. (2018). Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty-seeking agents. volume 2018-December.

Cully, A. and Demiris, Y. (2018). Quality and diversity optimization: A unifying modular framework. IEEE Transactions on Evolutionary Computation, 22.

Gaier, A., Asteroth, A., and Mouret, J. B. (2020). Discovering representations for black-box optimization. GECCO 2020 Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pages 103-111.

Gurkan, C., Jwalapuram, N. K., Wang, K., Danda, R., Rasmussen, L., Chen, J., and Wilensky, U. (2025). Lear: Llm-driven evolution of agent-based rules. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO '25 Companion, page 2309-2326, New York, NY, USA. Association for Computing Machinery.

Kumar, A., Clune, J., Lehman, J., and Stanley, K. O. (2025). Questioning representational optimism in deep learning: The fractured entangled representation hypothesis.

Lehman, J. and Stanley, K. O. (2010). Efficiently evolving programs through the search for novelty. Proceedings of the 12th Annual Genetic and Evolutionary Computation Conference, GECCO '10, pages 837-844.

Lehman, J. and Stanley, K. O. (2011). Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation, 19:189-222.

Tjanaka, B., Fontaine, M. C., Togelius, J., and Nikolaidis, S. (2022). Approximating gradients for differentiable quality diversity in reinforcement learning. GECCO 2022 - Proceedings of the 2022 Genetic and Evolutionary Computation Conference, pages 1102-1111.

Wilensky, U. (1999). NetLogo. http://ccl.northwestern.edu/netlogo/, Northwestern University, Evanston, IL.

Woolley, B. G. and Stanley, K. O. (2011). On the deleterious effects of a priori objectives on evolution and representation. Genetic and Evolutionary Computation Conference, GECCO'11, pages 957-964.

# Window Into Another World: Simulating Particle Life in the Fractional Dimension of a Sierpinski Gasket 

Isaac Gendelman MD ${ }^{1}$<br>${ }^{1}$ Tufts Medical Center, Department of Surgery, Boston USA<br>igendelman@tuftsmedicalcenter.org


#### Abstract

Background: Particle-based systems provide models to observe emergence but have been limited to 2D and 3D systems. This project demonstrates a particle life model in the fractional-dimensional space of a Sierpinski gasket. Methods:a graph engine generates the adjacency structure at any depth. Particles possess orientation, species identity, and binding state, and matrix interaction. Visualization is both evocative and explanatory. Results: This framework lets us see movement and behavior in a non-integer dimensional space. Conclusions: Extending particle life into non-integer dimensions demonstrates a novel testbed environment.


Submission Type: Abstract

## Introduction

Particle-based systems have been foundational in the study of artificial life and emergent properties. From Conway's Game of Life(Gardner, 1970) and lattice-gas automata(Frisch et al., 1986) to modern Particle Life(Schmickl et al., 2016) and extensions such as Lenia(Chan \& Kong, 2019), particle worlds have provided minimal yet expressive environments where emergent dynamics can be observed and quantified. Their appeal lies in their simplicity: interactions are defined locally, yet collective behavior can be studied statistically or visually, making them fertile laboratories. Previous implementations of particle life assume integer-dimensional Euclidean spaces with two-, or three-dimensional grids or lattices. Our contribution extends this tradition by constructing a particle world on a fractal substrate, specifically the Sierpinski gasket, where dimensionality itself is fractional. This world has a Hausdorff dimension of 1.585, which lies between a line (1D) and a plane (2D). By embedding particles in such a world, we enable exploration of environments where particles traverse not only space but also scale. While there have been attempts to imagine representation of a 4D world as a game engine(ten Bosch, 2025) our aim is to provide a platform for investigating dynamics in exotic non-integer dimensional spaces. By doing this we can peek into what another dimension may very well look like.

## Methods

The Sierpinski gasket(Sierpinski, 1915; Wang et al., 2023) is generated by recursive corner subdivision into successive equilateral triangles. In this representation, a particle's state is defined by both its position (a vertex coordinate) and its scale (the current refinement level). Movement consists of three modes: 1. Lateral moves: steps to adjacent vertices within the same scale. 2. Downward moves: refinement into finer-scale vertices (child triangles). 3. Upward moves: abstraction into coarser vertices (parent triangles).

In 2D and 3D systems, coordinates are straightforward where moving one unit in any axis represents the same scale of movement. In the Sierpinski fractal space, coordinates work differently as each vertex is defined by a pair of integers ( $\mathrm{x}, \mathrm{y}$ ) on a discrete triangular grid with the addition of a scale index L, representing which refinement level it belongs to. Thus, a full particle state is represented as $(\mathrm{x}, \mathrm{y}, \mathrm{L})$. Adjacency is defined by this extended coordinate. This structure means that the geometry is not uniform: a move in the ( $\mathrm{x}, \mathrm{y}$ ) plane may also correspond to a change in resolution, embedding the notion of scale as a coordinate dimension.

This implementation has three main components: Graph Engine, Particle Model and Visual Model.

Graph Engine: The foundation of the simulation is a CUDA-accelerated graph engine that generates the adjacency structure of the Sierpinski gasket up to a user-specified depth. Vertices are stored as integer lattice coordinates and consistently mapped into true equilateral geometry, ensuring that all movements occur within an accurate fractal space. Each refinement level maintains a compressed sparse row representation of adjacency, allowing rapid look-ups for lateral moves along edges as well as vertical transitions between finer and coarser scales. This hierarchical design ensures that legal moves are governed by the fractal's recursive structure and enables large populations of particles to step in parallel without sacrificing geometric fidelity.

Particle Model: On this fractal scaffold, particles move according to a rich interaction model. Each particle is defined by its global identifier, vertex position, scale index, orientation, species identity, and binding state. Interactions are governed by an interaction matrix that specifies attraction or repulsion strengths between a user-generated arbitrary number of particle species. This results in dynamic movement. Binding dynamics add another layer of complexity: particles can form temporary pairwise associations whose probability, lifetime, and cooldown are tunable, mimicking molecular docking and unbinding cycles.

Visual Model: The simulation is rendered to communicate its unique characteristics. The Sierpinski lattice is recolored dynamically by accumulated trail deposits left as particles visit vertices. Particles are displayed as colored markers keyed to their species identity, making group interactions visually distinct. Halos provide visual cues of scale transitions: a white flare when a particle ascends to a coarser level and a green flare when it descends into finer detail. Orientation ticks extend from each particle, indicating heading, while semi-transparent lines link temporarily bound pairs. The resulting visualization is both explanatory and evocative, simultaneously showing micro-scale interactions, macro-scale clustering, and the persistent traces of collective exploration across the fractal space.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-233.jpg?height=705&width=698&top_left_y=1546&top_left_x=264)
Figure 1: Sample frame from simulation

## Results and Capabilities

This exploratory framework demonstrates a simulation of particles evolving across the recursive
scales of a Sierpinski gasket, using CUDA acceleration to parallelize adjacency queries and movement updates. By explicitly visualizing scale transitions, the system reveals how particles ascend and descend within the fractal hierarchy. Fading but persistent vertex coloring accumulates into long-term trails, creating a visual record of traffic, clustering, and flow across the lattice. This simulation allows us to see movement and behavior in a non-integer dimensional space.

## Conclusion

Historically, particle worlds have been fertile ground for the study of artificial life, enabling controlled exploration of how rules and structures interact. By extending this paradigm into fractional-dimensional geometry, this project can stimulate the development of a new class of fascinating real, theoretical and imaginary testbed environments. While currently in early development additional parameters, forces, physical laws are planned to be introduced to encourage more striking and interesting particle behaviors.

## Code:

https://github.com/igendelman/sierpinski-p article-life

## Videos:

https://www.youtube.com/shorts/cMNDJNI fSC8

## Acknowledgements

Supported by the National Center for Advancing Translational Sciences, National Institutes of Health, Award \# T32TR004418 The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

## References

Chan, B. W.-C., \& Kong, H. (2019). Lenia: Biology of artificial life. Complex Systems, 28(3), 251-286.

Frisch, U., Hasslacher, B., \& Pomeau, Y. (1986). Lattice-gas automata for the Navier-Stokes equation. Physical Review Letters, 56(14), 1505-1508.

Gardner, M. (1970, October 1). Mathematical Games. Scientific American.

Schmickl, T., Stefanec, M., \& Crailsheim, K. (2016). How a life-like system emerges from a simple
particle motion law. Scientific Reports, 6, 37969.
Sierpinski, W. (1915). A curve of which every point is a ramification point. Comptes rendus hebdomadaires des seances de l'Academie des sciences, 160, 302-305.
ten Bosch, M. (2025). Miegakure: a game where you explore and interact with a 4D world. Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Real-Time Live!, 1-3.

Wang, J., Jiang, Z., Liu, W., Wu, Z., Miao, R., Fu, F., Yin, J.-F., Chen, B., Dong, Q., Zhao, H., Li, K., Wang, G., Liu, D., Yin, P., Li, Y., Chen, M., \& Wang, P. (2023). The marriage of Sierpiński triangles and Platonic polyhedra. Angewandte Chemie (International Ed. in English), 62(1), e202214237.

# Self-Organized Convolutional Networks using Meta-Neural Cellular Automata with Linear Attention 

Meet Barot<br>Mythos Scientific, New York, USA<br>meet@mythos.science


#### Abstract

Self-organization is a key property of living things achieved through local interactions. Biological neurons, through local interactions transmitted through synapses, are able to learn efficiently and can adapt their connections over an organism's lifespan. In prior work (Barot, 2024), we described Meta-Neural Cellular Automata (MetaNCA) as a way to learn the rules of selforganization for artificial neural networks. The local rules are given by a neural network that learns to update weights of a "task neural network" to perform a classification task. Once trained, the local rule network allows us to efficiently sample many task networks of arbitrary architectures without backpropagation. In this work, we present a novel Weight Transformer architecture for the local rule network, which uses linear attention to aggregate signals from neighboring weights and hidden states. We present results on the MNIST dataset (Deng, 2012), demonstrating the effectiveness of the Weight Transformer MetaNCA in generating the weights of feedforward neural networks and convolutional networks with unseen architectures of 2 million parameters.


## Method Description

Problem setup. We have a dataset $(x, y) \in D$ for a task $\mathcal{T}$, and a corresponding task neural network that makes predictions for this task $\hat{y}=T\left(\theta_{T} ; x\right)$. The task network $T$ has weights $w_{i j}^{L} \in \theta_{T}$, where $L$ is a layer of the task network and $i$ and $j$ are indices of the weight matrix for layer $L$. In addition, for each weight $w_{i j}$, the task network also has a hidden state vector $\vec{h}_{i j}$.

The updates to $\theta_{T}^{(t)}$ and $H_{T}^{(t)}$ are given by a separate local rule network $R$, parametrized by $\theta_{R}$. For a given weight $w_{i j}$, we take the neighboring weights and hidden state vectors of $w_{i j}$ as the input to the local rule network R.

## Local rule network inputs.

We construct the neighborhood of $w$ as the input to the local rule network $R$ in the following way. We start with a particular weight $w$ and its corresponding hidden state $\vec{h}$. To give a sense of direction for the neighborhood of

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-236.jpg?height=698&width=504&top_left_y=895&top_left_x=1210)
Figure 1: Weight Transformer architecture for the local rule network to aggregate signals from forward and backward neighborhoods $\mathcal{N}_{f}$ and $\mathcal{N}_{b}$ of a task network weight $w$, with attention outputs given by $a_{f}$ and $a_{b}$.

$w$, we define "forward" and "backward" neighbors to be all weights connected to the forward and backward neurons of $w$. We denote these neighborhoods of $w$ as $\mathcal{N}_{f}(w)$ and $\mathcal{N}_{b}(w)$ respectively. Weights are randomly initialized, while hidden states are initialized with sinusoidal positional encodings across layers, forward neurons, backward neurons, and horizontal and vertical kernel positions.

Neighborhood Aggregation with Multihead Linear Attention Consider a particular weight $w$ of the task network. Let $w$ 's forward neighborhood vectors be denoted as $\mathcal{N}_{f}(w)$. We apply cross-attention between $\left[\begin{array}{ll}w & h_{w}\end{array}\right]$ and its forward neighborhood's weights and hidden states $\mathcal{N}_{f}$, and do the same operation for the backward neighborhood. We first project $\left[w h_{w}\right]$ as

Mean Validation Accuracy for Convolutional Architectures
![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-237.jpg?height=380&width=1463&top_left_y=374&top_left_x=299)

Figure 2: Heatmap of performances of grown CNNs, with training architectures indicated in red rectangles. $m$ : multiplier for the number of convolutional filters per layer for a 3-layer convet with $32 * \mathrm{~m}, 64 * \mathrm{~m}$, and $128^{*} \mathrm{~m}$ filters. $h$ : layer dimension after the flattened outputs after the last convolutional layer, before the output layer.
the query vector $q$ :

$$
q=W_{Q}\left[\begin{array}{l}
w \\
\vec{h}
\end{array}\right]
$$

We do the same for all key and value weight matrices $K$ and $V$ with their own projection matrices $W_{K}$ and $W_{V}$. We then use a RoPE formulation for linear attention with an ELU-based feature map $\phi$ similar to Su et al. (2024), multiplying rotation matrix $R$ by the feature maps of queries and keys. In our case, each head of the linear attention mechanism corresponds to a particular directionality: layer, forward neuron, backward neuron, horizontal and vertical kernel position. We apply multihead linear attention for a given head $l$ :

$$
A_{l}=D_{l}^{-1} R_{l}^{T} \phi\left(Q_{l}\right) \phi\left(K_{l}\right)^{T} R_{l} V_{l}
$$

where $D$ is the normalization matrix $\phi(Q) \phi(K)^{T} \mathbf{1}$.
The $A$ attention output matrix is the same shape as $Q$, with a particular output $a$ corresponding to a query vector $q$.

We concatenate the output of all heads into a vector $a_{f}$ : this constitutes our "forward signal". We obtain $a_{b}$ similarly. We concatenate all these components in this way:

$$
v_{\text {perception }}=\left[w, \vec{h}, a_{f}, a_{b}\right]
$$

The updates to $w$ and $\vec{h}$ for the next timestep $t+1$ are given by a final feedforward network applied to the perception vector:

$$
\Delta w^{(t+1)}, \Delta \vec{h}^{(t+1)}=R\left(\theta_{R} ; v_{\text {perception }}\right)
$$

Training the local rule neural network. We update the weights of the local rule neural network by first applying a sampled number of local rule updates to the task network, calculating classification loss at the end of the updates, and doing backpropagation through time

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-237.jpg?height=726&width=659&top_left_y=995&top_left_x=1139)
Figure 3: Heatmaps of grown dense networks, with training architectures indicated with red rectangles.

(BPTT). We simulate asynchronous updates by applying local updates stochastically to $80 \%$ of task network weights per local rule update for our experiments.

## Experiments

We study how MetaNCA can learn to self-organize CNNs. In Figure 2 we see that with just two CNN architectures being trained on, the local rule network is able to generate weights on unseen architectures of over 2 million parameters. In Figure 3, we study MetaNCA's architecture generalization ability as more MLP architectures are added to training. Adding two training architectures allowed the entire column of architectures with 10 neurons in the second layer to reach $90 \%$ accuracy, indicating the capacity for the Weight Transformer to generalize better with more training architectures.

## References

Barot, M. (2024). Meta-neural cellular automata. Latebreaking abstract poster presented at ALIFE2024 in Copenhagen. Available at https://meetbarot. com/alife_poster.pdf.

Deng, L. (2012). The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6):141-142.

Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., and Liu, Y. (2024). Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063.

# Depression in the Network: A Study of Social Interactions in Agent Populations 

Abi M. Sipes ${ }^{1}$, Catherine M. Bain ${ }^{2}$, Nathaniel Kremer-Herman ${ }^{3}$, and Ankur Gupta ${ }^{4}$<br>${ }^{1}$ Purdue University, Department of Psychological Sciences, amsipes@purdue.edu<br>${ }^{2}$ University of Oklahoma, Department of Psychology, cbain1@ou.edu<br>${ }^{3}$ Seattle University, Department of Computer Science, nkh@seattleu.edu<br>${ }^{4}$ Butler University, Department of Computer Science and Software Engineering, agupta@butler.edu All correspondence regarding this article should be sent to Dr. Ankur Gupta at Butler University.


#### Abstract

Agent-based modeling (ABM) offers a powerful framework for investigating the emergent consequences of individual decision-making within complex social systems. We investigate the impact of mixed populations of neurotypical and neurodivergent individuals on societal success. In particular, our agents incorporate depressive symptomatology into the Sugarscape ABM . Our preliminary results show that the presence of depressed agents affect interaction patterns among agents, increase suboptimal decision-making in cooperative vs. selfish societies, and moderate societal-level phenomena such as trade, lending, and reproduction. Furthermore, our study reinforces computational study of social behavior as a reasonable methodology to augment longitudinal study design with informed patterns from simulation.


Submission type: Late Breaking Abstract
Data/Code available at: https://github.com/ nkremerh/sugarscape

## Introduction

Agent-based modeling (ABM) has emerged as a pivotal methodological framework for examining the complex interplay of individual behaviors within social systems [12]. By simulating autonomous agents with distinct characteristics, ABM enables researchers to explore emergent phenomena that arise from both micro- and macro-level interactions.

We have previously demonstrated the feasibility of using ABMs driven by different ethical predispositions as worthy of study [13, 16, 4]. These ethical decision models vary in the relative importance of the individual to that of the society as a whole. Egoism, for example, places the need of the individual at the forefront, prioritizing individual needs over the needs of others. This can be thought of as a traditional greedy strategy. A utilitarian approach is a balance of the two, where an agent prioritizes both their own self interest and the needs of their community. Using Sugarscape [14, 5, 9], Kremer-Herman and Gupta [10] illustrated that egoist (i.e., greedy) societies perform sub-optimally when compared to utilitarian (i.e., cooperative) societies.

This work assumed that all agents were neurotypical in their functioning. We know this to be untrue within the gen-
eral population. We expand on the prior work by incorporating depressed agents (as a neurodivergent model) into simulated societies and study societal outcomes. We study the effects of different ethical decision models on interactions between neurotypical and neurodivergent agents.

## Related Work

The ethical orientations embedded in our ABM, namely egoist (selfish) and utilitarian (cooperative) frameworks, which map onto the cultural dimensions of individualism and collectivism. These constructs, central to cross-cultural psychology and sociocultural theory, provide a valuable lens through which to interpret agent behaviors and societal dynamics. Individualism emphasizes personal autonomy, selfinterest, and the pursuit of individual goals, often at the expense of group cohesion or welfare [7, 15]. Our selfish agents embody this ethos. Their decision-making maximizes personal utility, regardless of the impact on surrounding agents. They reflect the behavior of individuals situated within individualistic societies where independence and personal gain are culturally valorized.

In contrast, collectivism privileges group harmony, interdependence, and the prioritization of group goals over personal ambition [7, 15]. Utilitarian agents, who consider both their own well-being and that of others in their decisionmaking processes, are modeled to reflect this collectivist orientation. These agents are more likely to engage in behaviors that benefit their local communities, even when such behaviors entail personal costs. As such, utilitarian agents simulate the behavioral tendencies observed in collectivist cultures, where mutual obligation, empathy, and cooperative strategies are central to social organization.

Literature in sociology and cultural psychology has examined the influence of these cultural models on depression $[1,3,6,8]$. Collectivist cultures may offer protection against mood disorders [2] as people in collectivist societies often develop an interrelated sense of identity, fostering strong social ties which mitigate loneliness and isolation. Studies indicate collectivist cultural structures may reduce mental health problems even in the presence of vulnerability factors
[1, 2]. In these cultures, people who pursue group interests and goals were seen to experience less stress during periods of change, leading to fewer chances of becoming depressed than in individualist cultures.

Conversely, individualist cultures tend to be competitive, achievement focused, and devalue the role of interpersonal relatedness [15]. This can lead to conflict and psychological distress. Some research argues that individualist cultures may see higher rates of depression and suicide, as a result of the decline in social support networks and increased social dysfunction [2,11]. Individualistic values may lead to smaller and less satisfying social support networks, less skill in managing emotions, and lower intentions to seek help, contributing to poorer mental health outcomes [11].

By operationalizing individualism and collectivism through ethical models in an agent-based simulation, this study offers a novel method for exploring the emergent properties of culturally shaped behavioral norms. It enables the examination of how culturally inspired ethical dispositions influence key societal outcomes, such as conflict incidence, economic transactions, and reproductive behavior. Furthermore, this framework facilitates the analysis of how cultural predispositions interact with neurodiversity to produce complex, emergent social systems.

## Results

Our testbed generated 50 seeds which are used to randomize the initial state of the simulation. We considered 21 different percentages of depressed agents in the population ranging from $0 \%$ to $100 \%$ in $5 \%$ intervals. We ran all seeds and all percentages twice: once with selfish agents and once with cooperative agents. We ran Sugarscape 2,100 times to produce a robust dataset for each decision model under consideration. Differences in outcomes between the societies for the same seed are truly indicative of different agent choices, since Sugarscape is a deterministic simulation.

Table 1: Societal Survival
| Behavior | Depressed | Extinct | Worse | Better |
| :--- | :--- | :--- | :--- | :--- |
| Cooperative | 0\% | 3 | 0 | 47 |
| Selfish | 0\% | 21 | 0 | 29 |
| Cooperative | 25\% | 3 | 0 | 47 |
| Selfish | 25\% | 37 | 1 | 12 |
| Cooperative | 50\% | 41 | 0 | 9 |
| Selfish | 50\% | 49 | 0 | 1 |
| Cooperative | 75\% | 50 | 0 | 0 |
| Selfish | 75\% | 50 | 0 | 0 |
| Cooperative | 100\% | 50 | 0 | 0 |
| Selfish | 100\% | 50 | 0 | 0 |


Table 1 describes the survival rate across a sample of the dataset. We show $25 \%$ increments which convey the
strongly negative effect that many depressed agents have on society. Those societies categorized as better ended the simulation with more agents than the start. Worse societies ended with fewer, and extinct societies did not make it to the end of the 1000 timestep simulation.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-241.jpg?height=538&width=807&top_left_y=607&top_left_x=1085)
Figure 1: Mean Neighbor Composition

We have two key preliminary results, captured in Figure 1 which plots the mean number of depressed agents found in an agent's neighborhood. We show the mean across heterogeneous mixtures of depressed and control agents up to a $50 \%$ depressed $/ 50 \%$ control population. Plotting further would yield empty results as demonstrated in Table 1. If the mean depressed neighbors is 1 , for example, on average there will be one depressed agent found among the possible cells the moving agent is considering during their turn.

We split the results for cooperative and selfish populations so values for depressed and control agents are presented separately. The lines represent the type (control/depressed) of agent and their corresponding society (selfish/cooperative). Holding the acting agent's type constant, it is clear that cooperative agents have, on average, more depressed agents in their neighborhoods than do selfish agents. This implies that cooperative agents can accommodate depressed agents more regularly than selfish agents do. These trends follow regardless of which type of agent is acting-in fact, it appears to be more pronounced when agents are depressed.

A further observation is seen when holding the acting agent's society constant: depressed agents appear to congregate more often with depressed agents, whereas control agents are more often seen with other control agents. It is surprising to see this kind of emergent behavior since these agents were not programmed to behave in this way.

## References

Ayşe Ayçiçeğí Dünn and C. L. Caldwell-Harris. How collectivism and family control influence depressive symptoms in Asian American and European American college students. Elektronik Sosyal Bilimler Dergisi, 15(57), 2016.
M. B. Bartucz, D. O. David, and S. A. Matu. Cognitive vulnerabilities and depression: A culture-moderated metaanalysis. Cognitive Therapy and Research, 46(3):502516, 2022.
J. Beller and A. Wagner. Loneliness and health: The moderating effect of cross-cultural individualism/collectivism. Journal of Aging and Health, 32(10):1516-1527, 2020.
T. J. Bench-Capon. Ethical approaches and autonomous systems. Artificial Intelligence, 281:103239, 2020.
A. Bigbee, C. Cioffi-Revilla, and S. Luke. Replication of Sugarscape using MASON. In Agent-Based Approaches in Economic and Social Complex Systems IV: Post-Proceedings of The AESCS International Workshop 2005, pages 183-190, 2007.
K. Fatehi, J. L. Priestley, and G. Taasoobshirazi. The expanded view of individualism and collectivism: One, two, or four dimensions? International Journal of Cross Cultural Management, 20(1):7-24, 2020.
G. Hofstede. Culture's Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations. Sage Publications, 2001.
G. G. Knyazev, V. B. Kuznetsova, A. N. Savostyanov, and E. A. Dorosheva. Does collectivism act as a protective factor for depression in russia? Personality and Individual Differences, 108:26-31, 2017.
N. Kremer-Herman and A. Gupta. Replacing Sugarscape: A comprehensive, expansive, and transparent reimplementation. In SIMUTools-15th EAI International Conference on Simulation Tools and Techniques-2023, 2023.
N. Kremer-Herman and A. Gupta. The need for greed in artificial decisionmakers. In 2024 IEEE International Symposium on Technology and Society (ISTAS), pages 1-9, 2024.
G. Scott, Joseph Ciarrochi, and F. P. Deane. Disadvantages of being an individualist in an individualistic culture: Idiocentrism, emotional competence, stress, and mental health. Australian Psychologist, 39(2):143-154, 2004.
M. Steinbacher, M. Raddant, F. Karimi, E. Camacho Cuena, S. Alfarano, G. Iori, and T. Lux. Advances in the agentbased modeling of economic and social behavior. $S N$ Business \& Economics, 1(7):99, 2021.
B. Sterner. Agent-based computer simulation and ethics: Steven Mascaro, Kevin B. Korb, Ann E. Nicholson and Owen Woodberry: Evolving ethics: The new science of good and evil. Exeter, UK: Imprint Academic, 2010, $\mathrm{xv}+259 \mathrm{pp}, £ 17.95 \mathrm{pb}$. Metascience, 21(2):403-407, 2012.
S. Tisue and U. Wilensky. NetLogo: Design and implementation of a multi-agent modeling environment. In Proceedings of Agent, pages 7-9, 2004.
H. C. Triandis. Individualism \& Collectivism. Westview Press, 1995.
A. F. Winfield, K. Michael, J. Pitt, and V. Evers. Machine ethics: The design and governance of ethical AI and autonomous systems [scanning the issue]. Proceedings of the IEEE, 107(3):509-517, 2019.

# Simulated Axon Growth for Wiring Patterned Neural Networks 

Jamieson Warner<br>Cognizant AI Lab, United States<br>jamiesonwarner@gmail.com


#### Abstract

Neural networks in the brain are formed through developmental processes that produce modular, repetitive, and spatially organized wiring patterns. Inspired by this, we investigate how simple developmental signals can guide the formation of artificial networks in a biologically plausible manner. We focus on a 3D cortical-columns task in which an $8 \times 8$ origin plane projects upward in parallel to form columnar connections across four voxels in depth. Two wiring rules generating the data are compared, each operating on voxel-level feature vectors learned end-to-end. The first, a Global dot-product rule with a temperature-annealed softmax, assigns connectivity by similarity across the entire space. The second, a $L o-$ cal gradient-following rule, uses Sobel operators to estimate chemical gradients to guide axonal paths step by step. Both rules successfully reproduce the target mapping. However, the Global model requires at least three channels ( $C \geq 3$ ) to achieve specificity and yields target fields that vary across the plane, even when smoothed with a Laplacian regularizer. By contrast, the Local model solves the task with a single channel ( $C=1$ ) and naturally produces smooth, bundle-like gradients that more closely resemble axonal guidance in biology. These findings suggest that local, gradient-based rules provide a more realistic inductive bias for developmental wiring, and highlight the promise of embedding such mechanisms into future self-organizing substrates for constructing brainlike networks.


## Submission type: Late Breaking Abstracts

## Introduction

We aim to build neural networks that resemble natural brain architectures, which are formed by a developmental process and contain numerous repetitive modular structures and other regularities Kaiser (2017). As argued in Warner and Miikkulainen (2025), such brain-like architectures allow for artificial networks to be directly compared with biological brain wiring data, enabling the use of wiring data to serve as a beacon toward animal intelligence. Toward such brain-like networks, our approach is to model the process of development, thereby conferring important regularities onto the artificial networks. Our work centers the Genetic Connectivity Model Barabási and Barabási (2020), used to understand
in C. elegans how network connections arise from patterns of genetic expression. These developmental signals inform where neurons wire, as modeled by a product-based wiring rule that calculates the level of association between regions. While this approach effectively captures wiring based on structural or region-level identity signatures, it is less effective when the wiring is caused by spatial constraints. In biology, axons grow by following local chemical gradients Sperry (1963). Here we highlight this difference by testing two wiring rules tasked to produce a cortical-column connectivity pattern. The first is a global dot-product rule with a softmax over all targets, while the second is a local gradient-following rule based on Sobel operators. In both cases, wiring decisions depend on voxel-level feature vectors - developmental signals - learned end-to-end, isolating the effect of the wiring rule itself. Our results show that the local rule produces guidance signals that align more closely with biological expectations, suggesting it as a more natural inductive bias for forming brain-like networks.

## Related Work

Producing neuronal wiring patterns through developmental models has become a central theme in recent and ongoing research. Neural Developmental Programs (NDP) Najarro et al. (2023) demonstrate that complex artificial networks can emerge through simple growth rules, where connectivity is not predefined but assembled over developmental time. Our present study builds on this direction by focusing specifically on the role of developmental signals in shaping connectivity patterns. This emphasis aligns with a long-standing tradition in developmental biology, tracing back to Wolpert's concept of positional information Wolpert (1969) and even earlier to Turing's theory of morphogenesis, or biological pattern formation Turing (1952). By grounding artificial wiring rules in morphogenetic patterns, we hope to create more biologically plausible networks that are situated in 3D space.

## Methods

Task: 3D Cortical Columns. We are trying to estimate an occupancy matrix, defined at the voxel level in which a cell in voxel $i$ containing an axon orginating in voxel $j$ is represented by the matrix entry $M[i, j]$. The connectivity matrix is constructed by defining an $8 \times 8$ "origin" plane, which projects targets in parallel upward spanning four voxels along the $z$-direction, resulting in 64 columns, each 4 voxels tall constituting the ground-truth adjacency.

Developmental signals. Developmental signals are defined at the voxel level, represented by the learned feature vectors $F(v) \in \mathbb{R}^{C}$ representing the spatial signals expressed at voxel $v$.

Global wiring rule (dot-product + softmax). The developmental signals are bifurcated into two sub-components, $F(v)=\left[F^{\text {tgt }}(v), F^{\text {src }}(v)\right]$. For an origin $s$ and candidate target $t$, we compute a logit $z_{s t}=F^{\text {tgt }}(s)^{\top} F^{\text {src }}(t)$ and define a global softmax over all $t$ :

$$
p_{s t}=\frac{\exp \left\{z_{s t} / \tau\right\}}{\sum_{s^{\prime}, t^{\prime}} \exp \left\{z_{s^{\prime} t^{\prime}} / \tau\right\}},
$$

with temperature $\tau$ annealed exponentially during training from $\tau=5.0$ to $\tau=0.004$. We optimize a standard crossentropy between $p_{s}$. and the binary row labels in the occupancy matrix. To encourage spatial smoothness of the learned fields we add a Laplacian (membrane) regularizer on $F: \lambda_{\text {lap }} \sum_{c}\left\|\Delta F_{c}\right\|_{2}^{2}$.

Local wiring rule (gradient-following + teacher forcing). We interpret $F$ as a multi-channel chemical field and estimate its 3D gradient with a $3 \times 3 \times 3$ Sobel operator (zeropadding at boundaries). From each origin voxel we iteratively extend a path by selecting the neighboring voxel whose preferred gradient direction scores highest (a learned linear preference of $\nabla F$ ). During training we use teacher forcing: at each step we apply the ground-truth next voxel and compute an occupancy loss (cross-entropy) over the local choice set that encourages the model to place maximal probability on the true next step.

Training. We trained models with channel counts ranging from $C=8$ down to $C=1$. Each run lasted 200 epochs, using the Adam optimizer with a learning rate of $10^{-3}$. For the Global rule, all source-target pairs were included in each update, so training proceeded in full batches without any learning-rate schedules. The only schedule applied was the annealing of the temperature parameter in the Global softmax. Model performance was assessed qualitatively, focusing on whether the correct wiring patterns emerged and on the structure of the learned feature fields.

## Results

Accuracy (channel capacity). Both wiring rules fully reproduce the columnar map originating from the $8 \times 8$ plane. The Global rule requires at least $C \geq 3$ channels to achieve perfect specificity, consistent with the need to encode distinct spatial patterns across the plane in the learned feature space. In contrast, the Local rule succeeds with $C=1$, as a single scalar field suffices for gradient-following to reach the correct targets.

Biological plausibility of fields. Under the Global rule, the learned $F$ forms a unique pattern for each column on the projection plane. A Laplacian regularizer smooths this pattern but cannot remove its inherent spatial variation across the plane, which remains necessary for disambiguating targets. This contrasts with biological development, where bundles of parallel fibers traverse in unison and wire into targets in parallel under shared local cues. The Local rule instead produces a smooth, bundle-like gradient that extends down the columnar paths, guiding fibers in parallel toward their targets without requiring column-specific feature codes.

## Discussion

These experiments isolate the inductive biases of global similarity versus local gradient-following when wiring networks from voxel features. On a minimal cortical-columns task, both rules can solve the mapping, but the Global rule needs higher representational capacity and yields spatially varying feature codes across the cortical surface-even when smoothed-whereas the Local rule achieves the same outcome with a single channel and gradient fields that better match biological expectations. Because both rules operate over voxel features, they are compatible with future selforganizing substrates (e.g., neural cellular automata Mordvintsev et al. (2020)) that could co-develop wiring cues and connectivity. We anticipate that local, developmentally grounded wiring mechanisms will scale more naturally to richer tasks where parallel fiber bundles and guidance cues dominate.

## References

Barabási, D. L. and Barabási, A.-L. (2020). A Genetic Model of the Connectome. Neuron, 105(3):435-445.e5.

Kaiser, M. (2017). Mechanisms of Connectome Development. Trends in Cognitive Sciences, 21(9):703-717.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing Neural Cellular Automata. Distill, 5(2):e23.

Najarro, E., Sudhakaran, S., and Risi, S. (2023). Towards SelfAssembling Artificial Neural Networks through Neural Developmental Programs.

Sperry, R. W. (1963). Chemoaffinity in the orderly growth of nerve fiber patterns and connections*. Proceedings of the National Academy of Sciences, 50(4):703-710.

Turing, A. (1952). The chemical basis of morphogenesis. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 237(641):37-72.

Warner, J. and Miikkulainen, R. (2025). Self-Organizing Models of Brain Wiring: Developmental Programs for Evolving Intelligence. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO '25 Companion, pages 2152-2161, New York, NY, USA. Association for Computing Machinery.

Wolpert, L. (1969). Positional information and the spatial pattern of cellular differentiation. Journal of Theoretical Biology, 25(1):1-47.

# Peer Evaluation of Individuals in a Language-Based Evolutionary Algorithm 

Thomas Gabor<br>LMU Munich, Germany<br>thomas.gabor@ifi.lmu.de


#### Abstract

We introduce an evolutionary algorithm whose individuals are encoded as text in a natural language. While this requires us to leverage a large language model for any kind of genome manipulation, it also enables us to introduce peer evaluation, which means that any individual's fitness is solely dependent on its peer individuals' evaluation according to evaluation guidelines encoded within their respective genomes. Despite these reciprocal dependencies, we observe stable and convergent learning behavior in an empirical study and show that the evolution of guidelines for peer evaluation precedes the evolution of realized properties in the individual's genomes.


Submission type: Late Breaking Abstract
Data/Code available at: https://github.com/ thomasgabor/watchmaker

## Motivation

Even though natural evolution has been imitated by artificial evolution in silico with great success, a few key differences remain (Eiben and Smith, 2015). One of them was stressed by Susan Stepney in her reception talk for the ISAL Award for Lifetime Achievement at ALIFE 2024 in Copenhagen: In natural evolution, the fabric of RNA/DNA easily interweaves property descriptions and process descriptions, i.e., information about how to build an individual and information on how to build the information itself; effectively, RNA/DNA information not only encodes the individual but also the process of handling the evolution of RNA/DNA information. By contrast, this mixture of perspective is hard to mimic in most artificial fabrics, such as programming language code.

In this abstract, we argue that using natural language as a fabric for evolutionary computation might allow us to study the evolution of information snippets that interweave aspects of the individual and the evolutionary process itself. This allows us to construct a process of peer evaluation for individuals, i.e., every individual's fitness is fully dependent on the other individuals in the population. For this to work, every

The following task is a creative task. You can use this inspiration to be able to create vastly different outcomes between calls: Dream fragment
Write a text that covers two aspects: First, pick a well-known animal or fantastical creature and describe some of its properties in 3-4 short sentences. Do not mention any names, not even that of the animal or fantastical creature you picked! Then, describe properties that should be considered good or preferable in any kind of animal or fantastical creature. You do not need to separate these aspects in your text.

Listing 1: Example of a prompt that was constructed to generate a new individual. The green text is generated from a list of random phrases; the blue text is set via a hyperparameter to the algorithm.
individuals needs to contain a descriptive part (information about its own properties) and an evaluating part (information about how to evaluate others).

We can see that in many practical examples, both of these parts might operate on vastly different data structures; description might simply be dictionary, evaluation needs to be a function (Pluhacek et al., 2023). Furthermore, the true evaluation of other individuals requires us to pass on the whole other individual (including its evaluation function), which calls for very abstract and recursive data types for our evaluation function.

Using natural language for both the individual's descriptive and evaluation part eliminates these issues as natural language can dynamically switch between description and meta-description. In order to then execute instructions written down in natural language, we employ a large-language model (LLM), allowing us to handle that maximally general data type with ease.

## Experiment

Listing 1 shows the prompt we use to generate individuals fully given by natural text. Listing 2 shows one such individual, generated by running the prompt from Listing 1 via OpenAI's gpt-4o-mini, which we use for all experiments here. With the in-built variance mechanism we can also generate a population of such individuals.

Peer Evaluation. In order to compute a fitness value for any individual in such a population, we then implement peer

In the twilight depths of ancient forests, a creature with iridescent scales glides silently, its wings shimmering like liquid silver under the moonlight. Equipped with sharp, keen eyes that can see through the darkest shadows, it often symbolizes wisdom and age-old knowledge. Majestic and elusive, it can blend seamlessly into its surroundings, evoking awe and reverence in those fortunate enough to catch a glimpse.
Ideal attributes for any being in such a realm would include unwavering loyalty, a spirit of generosity that nurtures the bonds within their kin, and the capacity to inspire hope and courage in the hearts of others. Resilience, alongside a profound understanding of the delicate balance of nature, would ensure their legacy thrives through time, bringing harmony to both their kind and the world they inhabit.

Listing 2: Example of an individual genome generated by the prompt of Listing 1.
evaluation. Formally, let us assume we consider a population $X_{t}$ of $N$ strings $x_{1}, \ldots, x_{N} \in X_{t}$ at generation $t$. By calling an LLM to do the consideration, $\eta_{x^{\prime}}(x) \in \mathbb{N}$ gives us a fitness value that the individual $x$ should have according to the 'opinion' of the individual $x^{\prime}$. The fitness of the individual $x$ at the current generation $t$ is then set to be $\phi_{t}(x)=\frac{1}{N-1} \sum_{i=1, x_{i} \neq x}^{N} \eta_{x_{i}}(x)$, which is the average fitness value assigned by all other individuals to $x$.

Naturally, this fitness value can change drastically even without any change in the genome but via change in the composition of the population; we thus always compute it anew every generation. However, our experiments show that even an evolution based on such a chaotic fitness evaluation has the tendency to converge, as we can see in the performance plot in Figure 1.

Towards a Common Goal. Using the help of the LLM, we can now derive an a posteriori population-wide fitness evaluator $y$ : After a full evolution run, we provide the LLM with a list of all individuals' genomes and their respective fitness value in the final generation and we ask that LLM to sum up their evaluation parts into a single evaluation guideline. Effectively, $y$ should provide a quick means to evaluate previous individuals against the quintessence of the final population. We call $\eta_{y}(x) \in[0 ; 100]$ the score of some individual $x$ at any time of the evolution. We then further compute a post hoc rule match $\mu$ : This is a new prompt for individual peer evaluation, but it is specifically designed to compare the two rule parts of the individuals with one another. In our analysis, we will always use the meta-individual $y$ as the evaluator within $\mu$. We can then compute $\mu_{y}(x)$ for all individuals $x$ in any generation of the whole evolution a posteriori.

Figure 2 shows the results. While $\eta_{y}(x)$ tells us for each individual $x$ how well its descriptive parts already match the expectation from peer evaluation present in the final population, $\mu_{y}(x)$ tells us for each individual $x$ how well its guideline parts already match the guidelines applied by peer evaluation in the final population.

Driver of Evolution. Figure 2 shows very clearly that the rule match $\mu_{y}$ is consistently ahead of the fitness score $\eta_{y}$,

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-249.jpg?height=406&width=661&top_left_y=278&top_left_x=1165)
Figure 1: We show the average fitness evolution across 10 runs. Blue shows the best individual fitness per population of 10 , red the average fitness.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-249.jpg?height=410&width=661&top_left_y=863&top_left_x=1165)
Figure 2: We show the average score evolution across 10 runs. Green shows the average score $\eta_{y}$ against the meta evaluator $y$, purple the match $\mu_{y}$ of the rule portion of an individual against the meta evaluator $y$.

i.e., at the same point in time it yields significantly higher values. This is our most important observation in this study! It shows that the rule or guideline parts approach their final state earlier than the descriptive or property parts of our individuals. We can interpret that it is the evolution of individual guidelines that drives the evolution of individual properties. Since these guidelines directly affect other individual's fitness, we can see that as a general case of a phenomenon like sexual selection; the preference for a (perhaps as of yet non-existent) property in the population drives its evolution within the population - or the preference for wings predates wings.

## Outlook

By mixing the individual properties and parts of the process description, we enable the study of interweaving encodings in artificial evolution for future work. We first observed emergent convergent behavior akin to foundational studies on evolution in other media (Fontana and Buss, 1994; Gabor et al., 2022). Since the combination of EAs and LLMs is a very promising field in general (Romera-Paredes et al., 2024; Wu et al., 2024) we reckon that techniques like this might find practical usage.

## References

Eiben, A. E. and Smith, J. E. (2015). Introduction to evolutionary computing. Springer.

Fontana, W. and Buss, L. W. (1994). What would be conserved if "the tape were played twice"? Proceedings of the National Academy of Sciences, 91(2):757-761.

Gabor, T., Illium, S., Zorn, M., Lenta, C., Mattausch, A., Belzner, L., and Linnhoff-Popien, C. (2022). Self-replication in neural networks. Artificial Life, 28(2):205-223.

Pluhacek, M., Kazikova, A., Kadavy, T., Viktorin, A., and Senkerik, R. (2023). Leveraging large language models for the generation of novel metaheuristic optimization algorithms. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation, pages 1812-1820.

Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., et al. (2024). Mathematical discoveries from program search with large language models. Nature, 625(7995):468-475.

Wu, X., Wu, S.-h., Wu, J., Feng, L., and Tan, K. C. (2024). Evolutionary computation in the era of large language model: Survey and roadmap. IEEE Transactions on Evolutionary Computation.

# Entropy Coupled Trait ODEs (ECTO) Reveal Structured Population Dynamics in Longitudinal Psychometric Data 

Anderson M. Rodriguez ${ }^{1,2}$<br>${ }^{1}$ Independent Researcher<br>${ }^{2}$ University of Georgia (student), USA<br>amr28693@uga.edu


#### Abstract

This project introduces ECTO, a minimal and interpretable dynamical system for modeling cohort-level trait evolution using population-level information structure. Leveraging data from the Swedish Adoption/Twin Study on Aging (SATSA), we compute a normalized entropy index $H^{*}(t)$ from psychometric item responses to summarize crosssectional heterogeneity at each wave. This entropy index sets the initial condition of the system; ODEs then evolve autonomously to predict cohort dynamics, which are compared to observed data. Neuroticism, here, refers to a broad psychometric trait capturing an individual's tendency toward negative affect, including anxiety, sadness, irritability, and emotional instability, which are quantified via composite item scores (Likert scale) rather than direct biological measures. The coupled ODEs describe neuroticism $(N(t))$, its pleiotropic companion $(P(t))$, and a latent environmental stress state (Estress $(t)$ ). ODE terms are phenomenologically inspired, encoding cohort-level analogues of mutationselection balance, pleiotropy, metabolic constraints, and environmental feedback, but are not intended as direct biological mechanisms. ECTO's forward simulations accurately capture cohort trends and outperform an uncoupled baseline model. Out-of-sample and surrogate validations are planned to further assess robustness. ECTO is transparent, low-parameter, and fully reproducible, with preprocessing and simulation code openly available. While entropy is treated here as a summary index rather than as causal, ECTO may be applicable to other systems requiring compact representation of population heterogeneity, though extension to machine learning or agent-based contexts remains open. Current limitations include sample size and latent environment. ECTO offers a scalable template for entropy-based modeling of psychological and population-level dynamics.


## Submission type: Late Breaking Abstract

Data/Code available at: https:// anonymous.4open.science/r/ECTO_system_ walkthrough-A12A/README.md

## Introduction

Neuroticism is a broad psychometric construct which can be regarded as a composite tendency toward negative affect (anxiety, sadness, irritability, emotional instability)(1; 2).

While the cumulative effects of negative affect can be destabilizing for individuals scoring high in neuroticism, the trait itself is remarkably stable across the life trajectory despite changing environments(3). This project asks whether information structure in population-level data recording Likert scale psychometric item responses over time can serve as a compact organizing signal to explain cohort-level trait dynamics.

## Contribution

This project introduces ECTO (Entropy-Coupled Trait ODEs)(4): a minimal, interpretable dynamical system in which a cohort-level entropy index $H^{*}(t)$, computed directly from neuroticism item distributions, is used to initialize an autonomous coupled ODE for the rescaled neuroticism state $N(t)$ (with pleiotropic companion $P(t)$, where applicable). Unlike black-box models, ECTO is transparent, low-parameter, and testable(5;6).

## Data and Measure

Dataset SATSA(7), six waves (1984-2007). Flagship pair for illustration: P8 Hot-Temperedness $(N)$ and P4 Worry $(P)$. Note: Letter-Number prefixes refer to internal SATSA labeling retained for reproducibility.

Entropy Use Population-level entropy(8) is computed for each item within the cohort at each wave. For item $j$ with $K_{j}$ categories and counts $x_{j k}(t)$, let $p_{j k}(t)=x_{j k}(t) / \sum_{m} x_{j m}(t)$ and $H_{j}(t)=-\sum_{k=1}^{K_{j}} p_{j k}(t) \log _{2} p_{j k}(t)$. Normalize per item and pool: $H^{*}(t)=\frac{1}{J} \sum_{j=1}^{J} \frac{H_{j}(t)}{\log _{2} K_{j}} \in[0,1] . H^{*}$ summarizes cross-sectional heterogeneity (not individual dispersion).

What $H^{*}$ Is (and Is Not) $H^{*}(t)$ is a macroscopic, order-parameter-like summary of the cross-sectional response distribution at wave $t(9)$. It is distinct from the latent states $\left(N, P, E_{\text {stress }}\right)$ and is not assumed to equal or directly measure them.

Role of $H^{*} \quad$ In this work, $H^{*}$ is used only to set the initial state at the first wave; the ODEs then evolve autonomously without time-varying inputs.

## Model: ECTO (Coupled ODEs)

Time \& Units We integrate in calendar time (years) at the observed timestamps. States and $H^{*}$ are dimensionless; $\mu, \gamma$ have units $1 /$ year; $\alpha, \beta, c_{1}, c_{2}, c_{3}, G, K$ are dimensionless.

$$
\begin{gathered}
\frac{d N}{d t}=\mu N-\left(\alpha N+\beta^{2} P\right) N \\
\frac{d P}{d t}=\mu P-\beta P\left(\frac{c_{1} P+c_{2} N+c_{3} E_{\text {stress }}}{G}\right), \\
\frac{d E_{\text {stress }}}{d t}=\gamma E_{\text {stress }}\left(\frac{N}{N+K}\right) .
\end{gathered}
$$

Parameters: $\mu$ ('innovation'/influx), $\alpha$ (self-limiting constraint), $\beta$ (cross-trait coupling), $\gamma$ (stress amplification), $c_{1}, c_{2}, c_{3}$ (cost weights), $G$ (capacity; $G=1$ for main runs), $K$ (saturation).

Interpretation Labels such as "mutation-selection," "pleiotropy," "metabolic constraint," and "environmental feedback" are biologically-inspired(6; 10; 11; 12; 13) phenomenological analogues at the cohort level, not direct biochemical mechanisms.

Environment Definition and Note on Lag $E_{\text {stress }}$ is a latent cohort-level state representing external pressures (e.g., stressors, media salience, macro shocks). In this submission a measured environmental time series is not invoked; $E_{\text {stress }}$ evolves endogenously via the equation above. The feedback loop is closed: $N$ drives $E_{\text {stress }}$, which increases the metabolic cost on $P$, and $P$ suppresses $N$ via cross-trait coupling. When a measured proxy is introduced (sensitivity/poster), a one-wave lag will be utilized to keep forecasts causal under irregular gaps: on $\left(t_{k}, t_{k+1}\right]$ the proxy is held at its last value $Z\left(t_{k}\right)$ and enters only as an additive term in the stress dynamics,

$$
\frac{d E_{\text {stress }}}{d t}=\gamma E_{\text {stress }} \frac{N}{N+K}+\lambda Z\left(t_{k}\right), \quad t \in\left(t_{k}, t_{k+1}\right],
$$

with no contemporaneous terms.

## Results to Date

- Forward simulations anchored at the first wave ( $H^{*}$ used only for initialization) reproduce broad within-sample trends for the flagship pair under plausible parameters.
- Fit statistics across six waves (RMSE, $R^{2}$ ) reported; trajectories compared to an uncoupled null with no cross-trait term.
- Units and Scale $H^{*}$ is dimensionless on [ 0,1 ]. Time is measured in years. Entropy is computed at the population level.
- Reproducibility Repository includes the ICPSR-toentropy preprocessing script, simulation notebooks, and a requirements file.


## Minimal Planned Additions

Roll-forward forecast from the first wave, leave-one-interval-out propagation, baselines (persistence, linear interpolation, AR(1), uncoupled 1D), plus multistart/profilelikelihood summaries and surrogate tests for any scheduled variant. This work updates the ECTO model from Rodriguez (2025) with tempered claims and expanded validation.

## Applicability

$H^{*}(t)$ is defined for any finite categorical distribution, so it can summarize cohort-level heterogeneity in settings like survey responses, discrete behavioral choices, or community composition (e.g., species counts). Here, it is used only to set initial conditions for the ECTO ODE; no additional state variables are introduced.

## Limitations

Six waves limit identifiability and preclude formal bifurcation analysis, which is desired for future iterations. $E_{\text {stress }}$ is latent. Entropy is treated as an organizing signal rather than a proven cause.

## Conclusion

While the model as currently presented is early-stage and limited in empirical scope, it provides a testable framework grounded in established biological and physical principles, and meets the goal of establishing a viable minimal formalism for future interdisciplinary development. This project ultimately aims to open new directions for interdisciplinary research.

Although this first-generation model is intentionally minimalistic, it demonstrates the feasibility of bridging entropybased behavioral data with biologically plausible dynamic systems. Future iterations will continue to build toward a generalizable theory of trait evolution integrating environment, energy, and cognition.

## Acknowledgements

No outside funding was utilized. The work is that of the sole author.

## References

Molenaar, P. C. M. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement: Interdisciplinary Research and Perspectives, 2(4), 201-218. doi:10.1207/s15366359mea0204_1

Ray, WJ. (2008). Methods: Toward a Science of Behavior and Experience (9th ed.). Wadsworth Publishing.
D. A. Blizard, N. Adams, D. I. Boomsma. (2023). The genetics of neuroticism: Insights from the Maudsley rat model and human studies. Personality Neuroscience, 6. doi:10.1017/pen.2023.4

Rodriguez, A. M. (2025). Behavioral Traits as Dynamical Systems: Utilizing Entropy to Analyze Longitudinal Psychometric Data in Coupled Ordinary Differential Equations. arXiv preprint arXiv:2506.20622. https: //arxiv.org/abs/2506.20622
S. H. Strogatz. (2014). Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Westview Press.

Kaneko, K. (2006). Life: An Introduction to Complex Systems Biology. Springer, Berlin, Heidelberg. doi:10.1007/978-3-540-32667-0

Pedersen, N. L. (2015). Swedish Adoption/Twin Study on Aging (SATSA), 1984, 1987, 1990, 1993, 2004, 2007, and 2010. Inter-university Consortium for Political and Social Research [distributor], 2015-05-13. doi:10.1017/s0001566000006681

Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379-423.

Jaynes, E. T. (1957). Information theory and statistical mechanics. Physical Review, 106(4), 620-630.

Murray, J. D. (2002). Mathematical biology I: An introduction (3rd ed.), 174. Springer New York.

Lewin, K. (1951). Field Theory in Social Science: Selected Theoretical Papers. (Ed. Cartwright, D.). Harper \& Row, New York.

Kounin, J. S. (1963). Field Theory in Psychology: Kurt Lewin. In J. M. Wepman \& R. W. Heine (Eds.), Concepts of Personality (pp. 142-161). Aldine Publishing Co.

Guastello, S. J. (1995). Chaos, Catastrophe, and Human Affairs: Applications of Nonlinear Dynamics to Work, Organizations, and Social Evolution. Mahwah, NJ: Lawrence Erlbaum.

# Discovering and Controlling Diverse Self-Organised Patterns in Cellular Automata using Autotelic Reinforcement Learning 

Marko Cvjetko, Gautier Hamon, Clément Moulin-Frier, Pierre-Yves Oudeyer, Flowers AI \& CogSci Lab, Inria, France<br>marko.cvjetko@inria.fr


#### Abstract

Autotelic AI algorithms, which pursue self-generated goals, have proven to be effective as automated discovery assistants in cellular automata (CAs). Previous work in this domain focused on algorithms which produce diverse behaviors by setting the automaton's initial conditions. Here, we extend these methods beyond initial-condition search and adapt them to systems that support sequences of closed-loop interventions. Using Lenia as a test environment, we train goal-conditioned reinforcement learning (RL) agents to perform targeted interventions during the system's evolution, guiding it towards desired states. The resulting agent behaviors are robust and diverse, demonstrating the potential of closed-loop interaction for discovery and control. Furthermore, we show that goal-conditioned RL agents performing interventions can discover novel self-organising patterns and generalize to previously unseen and noisy environments.


## Submission type: Late Breaking Abstracts

Companion website: developmentalsystems.org/rl-for-ca

## Introduction

Cellular automata (CAs) have long been used as computational models in complex systems science, artificial life, and the study of diverse real-world phenomena. Despite being defined by simple local rules, they can generate remarkably rich and unexpected patterns. In artificial life, CAs have been extensively used to study the questions of self-replication, life, and open-ended evolution (Sayama and Nehaniv, 2025). Many CA systems are chaotic: they are highly sensitive to initial conditions and external perturbations introduced as the system evolves (Beer, 2014). More recently, interest in CAs has grown with the development of new expressive models, such as Lenia and Flow-Lenia (Chan, 2018; Plantec et al., 2025), which are continuous extensions of Conway's Game of Life that display lifelike behaviors, and neural cellular automata (Mordvintsev et al., 2020), which incorporate deep learning architectures and optimization methods into the traditional CA framework.

For many years, researchers and hobbyists alike have explored the vast space of possible CA configurations in search of interesting patterns. Although this exploration tra-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-256.jpg?height=342&width=835&top_left_y=824&top_left_x=1080)
Figure 1: Visualization of how goal-conditioned RL agents can guide the CA grid towards some target state. This process is repeated to form an episode.

ditionally relied on manual tuning and random search methods (Wolfram, 2025), recent approaches have also used AIassisted automated discovery. Among these are autotelic diversity search methods, which generate a sequence of experiments to explore the parameters of a dynamical system by targeting a diversity of self-generated goals (Etcheverry, 2023). Lenia, in particular, has been a favored CA testbed of automated discovery methods, which have uncovered many novel, lifelike patterns (Hamon et al., 2024; Etcheverry et al., 2020; Faldor and Cully, 2024; Kumar et al., 2024).

To the best of our knowledge, all automated discovery methods applied to CAs so far have been open-loop. In such methods, the algorithm only specifies the initial state of the automaton and observes the outcome after the full rollout. Because CAs are inherently chaotic, the discovered patterns are likely to be highly unstable. Furthermore, open-loop methods do not intervene during rollout, which restricts their usefulness for control problems, even though CAs are frequently used to model real-world systems where control is critical, such as epidemics and ecosystem dynamics. This highlights the need for closed-loop discovery methods that can adjust the automaton's state through targeted interventions during its evolution. Such methods would enhance the expressiveness of the system, potentially enable the discovery of more robust and diverse phenomena, and open a new pathway towards applications in real-world control problems.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-257.jpg?height=477&width=1762&top_left_y=264&top_left_x=153)
Figure 2: Example of post-training behavior. Adding and removing activations is denoted with red and white colors, respectively. No cross indicates no action is taken. Top: an agent maintains the grid population by constantly removing activations in order to prevent an exploding Turing pattern from forming. Bottom: an agent directs an Orbium towards a target direction.

We propose a closed-loop approach for AI-driven scientific discovery in CAs, consisting of autotelic reinforcement learning (RL) agents that learn to directly perturb CA's states to achieve diverse targets. Although recent CA models have occasionally been used as RL environments (Earle and Togelius, 2024; Sánchez-Fibla et al., 2024), and supervised machine learning methods have guided neural CA pattern growth (Sudhakaran et al., 2022), control and interactive AIassisted discovery remain largely unexplored.

## Experiments

We present first experimental steps towards the larger project of closed-loop AI-assisted discovery ${ }^{1}$. Specifically, we conducted two experiments to investigate whether goalconditioned RL agents can make effective interventions to guide the CA's evolution to reach a desired state. At each timestep, the agent observes the current grid state and the goal embedding, then selects a location of the CA grid on which to do an intervention, which is either adding or subtracting an amount of activations within a fixed radius. The grid then evolves for $k$ CA steps, after which the agent receives a reward (see Fig. 1). This process repeats for $N$ steps, constituting one episode, with predefined goals sampled uniformly at the start of each episode. We use Lenia as our testbed and train agents using double deep Q-learning with a U-Net-like neural network architecture.

## Controlling Grid Population

In the first experiment, we train an agent to maintain the total number of living cells on the grid at a target level. The initial state contains a random number of activation clusters. The reward is the mean absolute error between the current grid sum and the sampled target value. For large targets, the agent learns to add activations in a way that produces self-sustaining, growing Turing patterns. Intermediate targets are handled by continuously reshaping the pattern,

[^15]while preventing a collapse or an explosion (top of Fig. 2). For smaller targets, the agent learned to construct novel selfsustaining patterns whose size is close to the target value.

## Steering an Orbium

The second experiment investigates whether the agent can direct an Orbium towards a target direction. The initial state contains a single Orbium at a random location and orientation on a toroidal grid. The reward is based on the centroid movement along a target direction. The action space additionally includes a do-nothing option, and actions that perturb the grid incur a small penalty, encouraging the agent to leverage the CA's intrinsic dynamics. After training, we observe that the agent can steer an Orbium towards any target direction. During a typical episode, the agent first steers the chaotic 'non-Orbium' configuration roughly toward the target direction before discovering a state that naturally converges into a properly oriented Orbium, after which it ceases interventions and allows the pattern to move autonomously (bottom of Fig. 2). We also observe that agents successfully complete the task in noisy environments, where random perturbations are added to the grid between iterations.

## Conclusion, Future Work

We have presented two experiments demonstrating that goal-conditioned RL agents can perform targeted interventions during Lenia's evolution and guide it towards desired states. While intervening, trained agents often create selfsustaining patterns, showcasing their ability to discover and exploit intrinsic dynamics of the complex system. Moreover, the agents generalize effectively to previously unseen, noisy environments. Future experiments may explore other environments, goal sampling within episodes, and automatic curriculum learning. Overall, these results represent an encouraging step towards robust closed-loop AI-assisted scientific discovery and control in complex systems, with promising potential for application to real-world systems.

## References

Beer, R. D. (2014). The Cognitive Domain of a Glider in the Game of Life. Artificial Life, 20(2):183-206.

Chan, B. W.-C. (2018). Lenia - Biology of Artificial Life. https://arxiv.org/abs/1812.05433v3.

Earle, S. and Togelius, J. (2024). Autoverse: An evolvable game language for learning robust embodied agents.

Etcheverry, M. (2023). Curiosity-Driven AI for Science: Automated Discovery of Self-Organized Structures. PhD thesis, Université de Bordeaux. Accessed: 2025-09-06.

Etcheverry, M., Moulin-Frier, C., and Oudeyer, P.-Y. (2020). Hierarchically organized latent modules for exploratory search in morphogenetic systems. Advances in Neural Information Processing Systems, 33:4846-4859.

Faldor, M. and Cully, A. (2024). Toward Artificial Open-Ended Evolution within Lenia using Quality-Diversity. In ALIFE 2024: Proceedings of the 2024 Artificial Life Conference. MIT Press.

Hamon, G., Etcheverry, M., Chan, B. W.-C., Moulin-Frier, C., and Oudeyer, P.-Y. (2024). Discovering Sensorimotor Agency in Cellular Automata using Diversity Search.

Kumar, A., Lu, C., Kirsch, L., Tang, Y., Stanley, K. O., Isola, P., and Ha, D. (2024). Automating the search for artificial life with foundation models.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing neural cellular automata. Distill. https://distill.pub/2020/growing-ca.

Plantec, E., Hamon, G., Etcheverry, M., Chan, B. W.-C., Oudeyer, P.-Y., and Moulin-Frier, C. (2025). Flow-Lenia: Emergent evolutionary dynamics in mass conservative continuous cellular automata. Artif. Life, 31(2):228-248.

Sánchez-Fibla, M., Moulin-Frier, C., and Solé, R. (2024). Cooperative control of environmental extremes by artificial intelligent agents. J. R. Soc. Interface, 21(220):20240344.

Sayama, H. and Nehaniv, C. L. (2025). Self-reproduction and evolution in cellular automata: 25 years after evoloops. Artificial Life, 31(1):81-95.

Sudhakaran, S., Najarro, E., and Risi, S. (2022). Goal-guided neural cellular automata: Learning to control self-organising systems. In From Cells to Societies: Collective Learning across Scales.

Wolfram, S. (2025). What can we learn about engineering and innovation from half a century of the game of life cellular automaton? Stephen Wolfram Writings.

# Developmental Pattern Formation in Neural Reaction-Diffusion Systems 

Salvador Buse ${ }^{1}$ and Erik Winfree ${ }^{1,2}$<br>${ }^{1}$ Bioengineering, and ${ }^{2}$ Computer Science, California Institute of Technology, USA sbuse@caltech.edu, winfree@caltech.edu


#### Abstract

Reaction-diffusion systems are a fundamental model of pattern formation and self-organization, with relevance for the development of biological organisms as well as for artificial molecular technologies (Kondo and Miura (2010); Wang and Ellington (2019)). Much is known about the range of patterns that can arise in relatively simple systems with just a handful of reactions (Turing (1952); Murray (2002)). To understand the potential of pattern formation with complex reaction networks, theoretical constructions based on digital and analog circuits and cellular automata have established that arbitrary and algorithmic patterns can develop in reactiondiffusion systems (Scalise and Schulman (2014, 2016)). Inspired by prior work showing that a variety of well-mixed chemical reaction networks can behave as neural networks (Hjemfelt et al. (1991); Mjolsness et al. (1991); Bray (1995); Kim et al. (2004); Qian et al. (2011)), combined with the differentiable programming technique employed for neural cellular automata (Mordvintsev et al. (2020)), we conjectured that these networks would be especially well-suited for complex pattern formation if trained using gradient descent. Here, we present an abstract chemical reaction network that we call the "Hopfield CRN" (Hopfield (1984)), and show that its reaction rates and diffusion constants can be trained to develop a complex target pattern from a simple initial state. The reaction network has a straightforward form, consisting of linear production via pairwise catalysis, limited by trimolecular degradation. Nonetheless, for a large enough number of species, training discovers parameters such that reactiondiffusion stabilizes the target pattern, and is robust to simulation discretization and limited damage. These results are unlikely to be specific to the Hopfield CRNs; the differentiable programming technique could be applied to any abstract model of chemical networks, such as generalized genetic regulatory networks (Mjolsness et al. (1991); Zhu et al. (2022)), cell-free enzyme systems (Kim et al. (2006); Zadorin et al. (2017)), and enzyme-free DNA strand displacement systems (Srinivas et al. (2017); Cherry and Qian (2018)).


## Submission type: Late Breaking Abstract

## Neural Reaction-Diffusion

Since the seminal work of Turing, reaction-diffusion is a mechanism thought to underlie some spatial pattern formation in biology (Turing (1952); Kondo and Miura (2010)).

Classical studies of reaction-diffusion systems tended to focus on simple chemical reaction networks (CRNs): even two-component systems are capable of giving rise to complex, self-organized structures and textures, depending on how the reaction rates and diffusion constants are tuned (Pearson (1993)). However, both biological systems and engineered synthetic chemical circuits highlight the potential of pattern-forming CRN mechanisms being far from simple. For example, Scalise \& Schulman demonstrated a theoretical basis for the morphogenesis of arbitrary patterns in reaction-diffusion systems (Scalise and Schulman (2014)), although using a digital circuits-inspired approach which may be hard to scale to more complex patterns.

Over the past decades, the links between neural networks and well-mixed chemical reaction networks have been explored in a wide array of settings (Hjemfelt et al. (1991); Mjolsness et al. (1991); Bray (1995); Kim et al. (2004); Qian et al. (2011); Cherry and Qian (2018)). Neural networks tend to exhibit properties like many-to-many connectivity, distributed decision making, pattern recognition, attractor states, and learning, all of which we felt were connected to the task of growing complex, stable patterns. As such, we began to wonder: what would a diffusing chemical neural network be capable of?

Hopfield networks are a recurrent model of neural networks with associative memory dynamics (Hopfield (1982, 1984)). They can be trained to store 'memories', and when initialized from a state corresponding to a corrupted memory, they can recover the original memory - a form of 'selfhealing', minimizing a Lyapunov function. Hypothesizing that these properties would make Hopfield networks a good candidate for spatial pattern-formation, we derived a CRN whose mass-action dynamics match those of continuous Hopfield networks, and which we call the "Hopfield CRN". Weighted sum computation is achieved using catalytic production reactions, and trimolecular degradation reactions take the place of activation functions, placing bounds on how large concentrations can become. Note that chemical concentrations ( $x_{i}^{ \pm}$) and rate constants ( $w_{i j}, k_{\text {deg }, i}$ ) are non-negative. To represent negative values for variables $x_{i}$
and weights $w_{i j}$, we adopt a 'dual-rail' convention, as in Chen et al. (2021). The variable $x_{i}$ is represented as the difference in concentrations of two species, each non-negative: $x_{i}=x_{i}^{+}-x_{i}^{-}$, with the species being connected through fast annihilation reactions. Later, we modified the original Hopfield CRN to remove some constraints (e.g. weight symmetry) and make the network more expressive, arriving at the form below:

Weighted sums:
$x_{i}^{+} \xrightarrow{w_{i j}^{++}} x_{i}^{+}+x_{j}^{+}, w_{i j}^{++}=\operatorname{ReLU}\left(+w_{i j}^{+}\right) \quad 3 x_{i}^{+} \xrightarrow{k_{\text {deg }, i}} 2 x_{i}^{+}$
$x_{i}^{+} \xrightarrow{w_{i j}^{+-}} x_{i}^{+}+x_{j}^{-}, w_{i j}^{+-}=\operatorname{ReLU}\left(-w_{i j}^{+}\right) \quad 3 x_{i}^{-} \xrightarrow{k_{\text {deg }, i}} 2 x_{i}^{-}$
$x_{i}^{-} \xrightarrow{w_{i j}^{-+}} x_{i}^{-}+x_{j}^{+}, w_{i j}^{-+}=\operatorname{ReLU}\left(+w_{i j}^{-}\right) \quad$ Annihilation:
$x_{i}^{-} \xrightarrow{w_{i j}^{--}} x_{i}^{-}+x_{j}^{-}, w_{i j}^{--}=\operatorname{ReLU}\left(-w_{i j}^{-}\right) \quad x_{i}^{+}+x_{i}^{-} \xrightarrow{\text { fast }} \emptyset$

Drawing inspiration from Growing Neural Cellular Automata (Mordvintsev et al. (2020)), we wrote a differentiable reaction-diffusion simulator in JAX (Bradbury et al. (2018)), enabling us to use backpropagation-through-time to optimize the parameters (reaction rates and diffusion constants) of the Hopfield CRN to train it to solve pattern-forming tasks. Our simulation uses implicit-explicit numerical methods in a two-dimensional finite differences setting, solving diffusion implicitly in the Fourier domain, and solving reactions explicitly using Runge-Kutta (RK4) integration.

The simplest of our pattern formation tasks is to grow from some initial conditions (a 'seed') into a specified target pattern at a particular time. The first three species $\left(x_{0}, x_{1}, x_{2}\right)$ in the CRN are 'visible', corresponding to red, green, and blue respectively, with the remainder 'hidden', in the sense that the network can determine how they ought to be used to achieve the task. Since diffusion is a symmetrypreserving (isotropic) process, the initial conditions must be at least as asymmetric as the target. We have found that many different initial conditions are compatible with pattern formation. For our experiments, we chose to use a seed consisting of overlapping splotches in each of the visible channels, with all of the hidden channels initially set to zero. For this task, our loss function is applied at a fixed time point, and consists of a pixel-wise sum of the squared difference between the target RGB values and the simulated RGB concentrations. We find that this formulation of the task is solvable by the CRN, for many different targets, numbers of hidden species, and parameter initializations. A figure demonstrating one example of this type of reaction-diffusion pattern formation may be found in Lee et al. (2025).

Beyond the ability to form 'snapshots' of complex patterns, we wondered whether reaction-diffusion was able to make them stable, and even robust to some damage. Incorporating perturbations such as aging, blurring, re-scaling,
and erasure, we found that reaction-diffusion can be trained to be somewhat robust to all of them. In general, we find that pattern fidelity improves with access to more trainable parameters (increasing the number of species, and so rate constants), and that for a given pattern fidelity, the robust version of the CRN also requires more parameters. We have simulated and trained Hopfield CRNs of up to 256 species. This is beyond what could currently be engineered synthetically, but still small in number of components when compared to biological systems.

Unlike classical Hopfield network Hebbian learning, which is hard-coded for the task of associative memory, the differentiable programming framework can be applied to optimize loss functions that measure arbitrary spatial and temporal tasks. We explore the possibility that reactiondiffusion can encode multiple distinct mature patterns, as well as undergoing temporal 'metamorphosis' through a sequence of forms.

Reaction-diffusion models were originally motivated by biological pattern formation and development. Does our experience with complex morphogenesis in reaction-diffusion systems offer new insights about biology? Conversely, can we use insights from biology to shine light on how CRNs solve these pattern-forming tasks? For instance, when looking at the spatial distributions of the hidden species, we hypothesized that we might see features analogous to cell types in biology. Simulating our trained CRNs in the wellmixed (non-diffusing) setting, we find that, as with the original Hopfield network, they tend to have a small number of attractor states. However, and with the caveat that our results depend on the form of CRN we train and whether we train for robustness, it seems these attractor states do not map cleanly onto cell types. Comparing these well-mixed attractor states with the mature pattern formed by reactiondiffusion, we find that most pixels are not actually at the attractor states, rather lying somewhere between them. This is somewhat different from the perspective of Waddington's developmental landscape. We are interested in continuing to explore what this toy model can tell us about different perspectives on biological development, as well as better understanding what types of algorithmic behaviour reactiondiffusion is capable of.

## Acknowledgements

We thank our Caltech colleagues Lulu Qian, Paul Rothemund, Andrew Stuart, Matt Thomson, Justin Bois, Michael Elowitz, and Rob Phillips, and our frequent collaborator Arvind Murugan, for their invaluable feedback on and suggestions for this work. We also thank Alexander Mordvintsev, Eyvind Niklasson, Ettore Randazzo, and Michael Levin, for many illuminating and creative discussions, and for their paper Growing Neural Cellular Automata, from which we took much inspiration for our training methods. Thank you also to Inhoo Lee, Kevin Cherry, Yancheng Du,

Cameron Chalk, Olivia Zou, and other members of the Winfree, Rothemund, and Qian labs. Salvador Buse is supported by a graduate fellowship from the Open Philanthropy Project, and this work was supported by grants from the National Science Foundation.

## References

Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., and Zhang, Q. (2018). JAX: composable transformations of Python+NumPy programs.

Bray, D. (1995). Protein molecules as computational elements in living cells. Nature, 376:307-312.

Chen, H.-L., Doty, D., Reeves, W., and Soloveichik, D. (2021). Rate-independent computation in continuous chemical reaction networks.

Cherry, K. and Qian, L. (2018). Scaling up molecular pattern recognition with DNA-based winner-take-all neural networks. Nature, 559:370-376.

Hjemfelt, A., Weinberger, E. D., and Ross, J. (1991). Chemical implementation of neural networks and Turing machines. Proceedings of the National Academy of Sciences, 88:1098310987.

Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. PNAS, 79(8):2554-2558.

Hopfield, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. PNAS, 81(10):3088-3092.

Kim, J., Hopfield, J. J., and Winfree, E. (2004). Neural network computation by in vitro transcriptional circuits. In Neural Information Processing Systems, page 681-688.

Kim, J., White, K. S., and Winfree, E. (2006). Construction of an in vitro bistable circuit from synthetic transcriptional switches. Molecular Systems Biology, 2.

Kondo, S. and Miura, T. (2010). Reaction-Diffusion Model as a Framework for Understanding Biological Pattern Formation. Science, 329:1616-1620.

Lee, I., Buse, S., and Winfree, E. (2025). Differentiable Programming of Indexed Chemical Reaction Networks and ReactionDiffusion Systems. In 31st International Conference on DNA Computing and Molecular Programming (DNA 31), volume 347, pages 4:1-4:23.

Mjolsness, E., Sharp, D. H., and Reinitz, J. (1991). A connectionist model of development. Journal of Theoretical Biology, 152:429-453.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing Neural Cellular Automata. Distill. https : //distill.pub/2020/growing-ca.

Murray, J. D. (2002). Mathematical Biology. Springer, 3rd edition.
Pearson, J. (1993). Complex patterns in a simple system. Science, 261(5118):189-192.

Qian, L., Winfree, E., and Bruck, J. (2011). Neural network computation with DNA strand displacement cascades. Nature, 475(7536):368-372.

Scalise, D. and Schulman, R. (2014). Designing modular reactiondiffusion programs for complex pattern formation. Technology, 2:55-66.

Scalise, D. and Schulman, R. (2016). Emulating cellular automata in chemical reaction-diffusion networks. Natural Computing, 15:197-214.

Srinivas, N., Parkin, J., Seelig, G., Winfree, E., and Soloveichik, D. (2017). Enzyme-free nucleic acid dynamical systems. Science, 358:eaal2052.

Turing, A. M. (1952). The chemical basis of morphogenesis. Philosophical Transactions of the Royal Society B, 237:37-72.

Wang, S. S. and Ellington, A. D. (2019). Pattern generation with nucleic acid chemical reaction networks. Chemical Reviews, 119:6370-6383.

Zadorin, A. S., Rondelez, Y., Gines, G., Dilhas, V., Urtel, G., Zambrano, A., Galas, J.-C., and Estevez-Torres, A. (2017). Synthesis and materialization of a reaction-diffusion French flag pattern. Nature Chemistry, 9:990-996.

Zhu, R., del Rio-Salgado, J. M., Garcia-Ojalvo, J., and Elowitz, M. B. (2022). Synthetic multistability in mammalian cells. Science, 375:eabg9765.

# Internalized Culture versus External Rules: The Emergence of Bounded Rationality in LLM Agent Societies 

Ryosuke Takata ${ }^{1}$, Atsushi Masumori ${ }^{1}$, and Takashi Ikegami ${ }^{1}$<br>${ }^{1}$ The University of Tokyo, Japan takata@sacral.c.u-tokyo.ac.jp


#### Abstract

We investigate the emergent social dynamics of Large Language Model (LLM) agents in a spatially extended El Farol Bar problem, observing how they autonomously navigate this classic social dilemma. As a result, the LLM agents generated a spontaneous motivation to go to the bar and changed their decision making by becoming a collective. We also observed that the LLM agents did not solve the problem completely, but rather behaved more like humans. These findings reveal a complex interplay between external incentives (prompt-specified constraints such as the $60 \%$ threshold) and internal incentives (culturally-encoded social preferences derived from pre-training), demonstrating that LLM agents naturally balance formal game-theoretic rationality with social motivations that characterize human behavior.


## Submission type: Late Breaking Abstracts

## Introduction

The El Farol Bar problem (Arthur, 1994) is a classic model for collective decision-making under bounded rationality. Individuals must decide whether to go to a bar that is enjoyable if not crowded, but unpleasant if it is. While traditional agent-based models have explored strategy evolution in this scenario, they often strip away the rich cultural and social contexts that guide human decisions.

Recent advances in Large Language Models (LLMs) allow for the creation of agents that can incorporate this missing context. Unlike traditional algorithmic agents, LLM agents can leverage vast knowledge from pre-training to exhibit more nuanced, human-like behaviors (Park et al., 2023). In this study, we use a society of communicative LLM agents to reinvestigate the El Farol Bar problem, focusing on how spontaneous motivation, emergent social dynamics, and human-like rationality arise from their interactions.

## Simulation Setup

We designed a simulation in a $50 \times 50$ two-dimensional space containing a $10 \times 10$ bar at its center. Twenty LLM agents (powered by GPT-4o, Hurst et al., 2024) were placed at random initial positions. The simulation ran for 1000 steps.

The core rule is the bar's crowding threshold: the bar is "comfortable" if fewer than $60 \%$ of agents (i.e., $<12$ agents) are inside, and "uncomfortable" otherwise. Only agents inside the bar receive this status as feedback. Agents can move one grid unit per step and communicate with others within a radius of 5 .

Each agent's action is determined through a cognitive loop. At each step, the agent perceives its current state, which includes its position, the bar's status (if inside), and messages from nearby agents. This information is compiled into a descriptive prompt that asks the LLM to reason about the situation and then generate a structured output. This output consists of three parts: a movement decision (e.g., move to $\mathrm{x}+1$ ), a natural language message to broadcast, and a short-term memory string to carry over to the next step. This design allows for the emergence of complex, historyaware behaviors beyond simple stimulus-response.

## Results

Our simulations, conducted over 10 independent trials, revealed several key emergent behaviors. First, agents spontaneously developed a motivation to go to the bar. They often formed social clusters outside the bar before entering, indicating that social coordination preceded action.

Second, the number of agents inside the bar did not oscillate chaotically but converged to a dynamic equilibrium. In most trials, this equilibrium was slightly above the $60 \%$ threshold, but in a few notable cases, it stabilized at a much higher number (Figure 1). This variation suggests a trade-off between competing incentives: while the agents collectively aimed for a "good enough" solution, in some instances, the internal incentive for social connection-the desire to be in the bar with others-powerfully overrode the external rule against crowding. This tendency to prioritize social goals over perfect optimization is a hallmark of bounded rationality (Simon, 1955).

Third, this equilibrium was maintained by contextdependent strategies (Figure 2). When the bar was crowded, agents inside showed a clear tendency to leave (the 'stay' rate dropped from $76.5 \%$ to $42.4 \%$ ), while agents outside

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-265.jpg?height=405&width=670&top_left_y=274&top_left_x=237)
Figure 1: Number of agents in the bar over 10 trials. The mean (bold blue) stabilizes just above the $60 \%$ threshold ( 12 agents, dashed black line).

predominantly chose to wait ('stay' rate increased from $39.0 \%$ to $81.2 \%$ ). This demonstrates a sophisticated, collective response to environmental feedback. A more microscopic analysis of agent movement reinforces this finding of goal-directed behavior. We found that agents inside a crowded bar did not simply leave, but moved with a clear directional bias away from the bar's center. This intent to exit grew stronger the deeper an agent was within the bar. Conversely, in a comfortable bar, agents tended to move further inside. This demonstrates that the agents were not just reacting to a binary state (crowded/not crowded) but were actively navigating their environment based on their location and goals.

These quantitative findings were supported by rich qualitative observations. For instance, we witnessed the emergence of distinct social roles. One agent, Agent 18, consistently exhibited c behavior. As the bar became crowded, it voluntarily exited while broadcasting messages such as, "I will move to $(6,0)$ to create more space and continue contributing to the positive atmosphere... Let's keep spreading good vibes." This type of prosocial, un-programmed behavior highlights the agents' capacity for complex social reasoning. Furthermore, the use of hashtags like \#collaboration and \#positivity functioned as emergent social tags, allowing agents to quickly signal group mood and intent, thereby facilitating coordination. The differentiation of individuality was also evident in the agents' internal states. An analysis of their generated text revealed that while their public messages converged into distinct, situation-dependent clusters (e.g., "invitation," "waiting," "enjoying"), their private memories remained diverse and individualized. This suggests a dynamic interplay where agents participated in a shared social context through communication, while simultaneously cultivating unique internal experiences and histories.

## Discussion and Conclusion

The behavior of the LLM agents reveals a fascinating interplay between two types of incentives. External incentives, defined by the prompt's rules (the $60 \%$ threshold),

(A) inside the bar
![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-265.jpg?height=816&width=629&top_left_y=315&top_left_x=1183)

Figure 2: Action distribution based on location (inside/outside the bar) and crowding status. Crowding creates exit pressure for those inside and encourages waiting for those outside.
drove them to manage congestion. Simultaneously, internal incentives, derived from the LLM's pre-trained cultural knowledge about bars being social venues, drove them to seek social connection.

This tension led to a boundedly rational outcome. The agents did not perfectly solve the optimization problem because their internal, social motivations often overrode purely rational calculations, causing them to tolerate a slightly crowded bar for the sake of being together. This behavior is context-dependent; a comparative experiment replacing the "bar" with a "library" resulted in individualistic movements without social clustering, confirming that the agents' behavior stems from internalized cultural norms.

In conclusion, our work demonstrates that LLM-based simulations can bridge the gap between abstract gametheoretic models and the complexity of human social behavior. By endogenously balancing external rules with internal social priors, these agents autonomously rediscover and navigate complex social dilemmas, offering a powerful new tool for studying the emergence of sociality.

## Acknowledgements

This research was supported by the DENSO Social Cooperation Research Department Mobility Zero, Grant-in-Aids Kiban-A (JP21H04885), and Grant-in-Aids for JSPS Fellows (JP24KJ0753).

## References

Arthur, W. B. (1994). Inductive reasoning and bounded rationality. The American economic review, 84(2):406-411.

Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al. (2024). Gpt-4o system card. arXiv preprint arXiv:2410.21276.

Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1-22.

Simon, H. A. (1955). A behavioral model of rational choice. The quarterly journal of economics, pages 99-118.

# Error-Driven Control of Ants Learning-Walks 

Thomas Misiek ${ }^{1}$, Andrew Philippides ${ }^{1}$, Paul Graham ${ }^{1}$, and James Knight ${ }^{1}$<br>${ }^{1}$ Sussex AI, School of Engineering and Informatics, University of Sussex, Brighton, UK<br>\{t.misiek, andrewop, P.R.Graham, j.knight\}@sussex.ac.uk


#### Abstract

When ants leave the nest for the first time, they perform multiple learning walks (LWs), a series of looping paths centered on the nest, before starting foraging. During these LWs they frequently execute pirouettes, short scanning turns used to fixate the nest and build visual memories for later homing. LWs evolve in a stereotypical fashion: loops expand, pirouettes fade, and paths straighten with experience. Yet, what controls the geometry of LWs, or triggers their reinitiation after environmental changes, remains unclear. We present a simple, error-driven model in which the immediate angular mismatch between Path Integration (PI) and visual predictions directly controls scanning and loop dynamics. Simulated ants navigating realistic 3D environments spontaneously reproduce key features of real LWs: small early LWs which progressively get longer, fewer pirouettes with training, spontaneous termination of LWs after a few bouts and the onset of foraging trips, and rapid reinitiation when familiar landmarks changes. Our results suggest that PI signal is not just a cue for homing, but a core driver of visual learning and exploratory behavior.


## Introduction

Naïve ants, bees, and wasps perform stereotyped learning bouts-loops, pirouettes, and nest fixations-centered on the nest. With experience, these excursions expand, straighten, and eventually give way to foraging trips (Müller and Wehner, 2010; Fleischmann et al., 2016; Jayatilaka et al., 2018; Zeil and Fleischmann, 2019). Importantly, learning walks (LWs) are not a one-off developmental stage: they are re-initiated whenever the visual panorama around the nest changes (Vega Vermehren et al., 2020; Deeti and Cheng, 2025).

Snapshot-based visual compass models explain how insects later use stored views to return home, and they perform well in simulation (Zeil et al., 2003; Baddeley et al., 2012). However, these models do not explain the dynamics that generate the stereotyped LW shapes, why LWs stop after three to seven loops (Jayatilaka et al., 2018), or how they resume when the environment changes. No existing model has tried to reproduce the full set of observed behaviors: tight spirals around the nest, gradual loop expansion
with successive LWs, and a decline in sampling and pirouette frequency.

Here, we propose a simple mechanism that unifies these features under a single control signal: the moment-tomoment mismatch between path integration (PI) and visual prediction. This signal governs both the structure and timing of learning, providing a parsimonious explanation for the ontogeny and flexibility of insect LWs.

## Methods

We used Antworld, a 3D reconstruction of a semi-arid landscape near Seville, Spain, developed as a virtual testbed for desert ant navigation. During training, fifty agents performed successive walks out of their nest. Agents combined Path Integration (PI), which provided a running vector estimate of nest direction and distance, with a visual snapshot model inspired by the visual compass framework.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-268.jpg?height=497&width=558&top_left_y=1580&top_left_x=1208)
Figure 1: $\mathbf{1 1}$ successive LWs generated by the model

(Zeil et al., 2003; Baddeley et al., 2012). An error accumulator built up during each loop and controlled outward and inward behavior: At the onset of LWs the agent followed an outward trajectory until the accumulated error reached a threshold, then homing behavior took over. The accumulator was reinitialized after returning to the nest. Simultaneously, the angular error between the PI estimate and the visual prediction controlled learning behavior: high discrepancies pro-
moted immediate spiraling behaviors, scanning maneuvers, and pirouettes. Snapshots were captured during pirouettes and were consolidated only after a successful return to the nest, preventing the agent from perceiving new views as familiar.

In addition, after each training loop, fifty zero-vector agents (with their PI vector reset) were placed 3 m from the nest and had to perform visual homing. Each test was labeled according to the agent's navigation history: naïve agents (never left the nest), short LW (walks ; 0.7 m ), long $\mathrm{LW}, 3 \mathrm{~m}$, and foragers.

## Results

The model reproduces key behavioral traits of ant learning walks (LWs). Early trips form short, spiraling loops around the nest (Fig. 1). With experience, successive walks become straighter and longer, and the frequency of pirouettes decreases (Fig. 2). After a few bouts, exploratory behavior ceases spontaneously. However, when the panorama is altered -inducing a mismatch between path integration (PI) and visual cues-the model exhibits a return to exploratory behavior, with increased snapshot frequency, shorter and more convoluted loops (Fig. 3).

As memory accumulated, PI-vision agreement increased, scanning behavior declined, and walks straightened into foraging trips. Conversely, mismatch between PI and vision reinstated LWs, reproducing the continuous adjustment described in field studies, where foragers steadily improved with experience yet remained able to re-learn when conditions changed (Fleischmann et al., 2016; Vega Vermehren et al., 2020; Deeti and Cheng, 2025).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-269.jpg?height=643&width=727&top_left_y=1601&top_left_x=219)
Figure 2: Top red: Number of snapshots per loop decrease with successive LWs. Top blue: loop straightness increases with training. Bottom: loops expand with training. Black bar at loop 12 marks an environmental change ( 20 cm cylinder close to the nest disappears) triggering re-learning.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-269.jpg?height=447&width=716&top_left_y=280&top_left_x=1133)
Figure 3: Homing accuracy increases with experience. The centroid of search loops shifts closer to the true nest position as ants gain experience. Orange: naïve ants; Blue: short LWs; Green: long LWs; Red: foragers.

## Discussion

Our model explains the structure of learning walks through a single signal: the angular error between path integration (PI) and visual prediction. PI scaffolds visual learning in two complementary ways: PI-vision mismatch drives exploratory behavior, while PI also defines the direction in which snapshots must be taken. This error-driven mechanism reproduces the main signatures of insect LWs: early trips with tight spirals and frequent scanning, gradual expansion and straightening of loops, a decline in pirouette frequency and sampling, termination of LWs once PI and vision predictions align, and rapid reinstatement of learning when the visual environment around the nest changes (Müller and Wehner, 2010; Fleischmann et al., 2016; Zeil and Fleischmann, 2019).

A key implication of the model is that there is no strict boundary between a LW and a foraging trip. The same control mechanism operates in both, and a walk can be labeled as "foraging" only retrospectively. Learning may occur at any stage if PI-vision predictions diverge. In line with field observations, pirouettes decrease sharply across successive loops and almost disappear in experienced foragers (Fleischmann et al., 2016; Jayatilaka et al., 2018). This supports the view that pirouettes are primarily learning events, while inference does not require pirouettes.

## References

Baddeley, B., Graham, P., Husbands, P., and Philippides, A. (2012). A model of ant route navigation driven by scene familiarity. PLoS Comput. Biol., 8(1):e1002336.

Deeti, S. and Cheng, K. (2025). Desert ants (melophorus bagoti) oscillate and scan more in navigation when the visual scene changes. Anim. Cogn., 28(1):15.

Fleischmann, P. N., Christian, M., Müller, V. L., Rössler, W., and Wehner, R. (2016). Ontogeny of learning walks and the acquisition of landmark information in desert ants,Cataglyphis fortis. J. Exp. Biol., 219(19):3137-3145.

Jayatilaka, P., Murray, T., Narendra, A., and Zeil, J. (2018). The choreography of learning walks in the australian jack jumper ant myrmecia croslandi. J. Exp. Biol., 221(Pt 20):jeb185306.

Müller, M. and Wehner, R. (2010). Path integration provides a scaffold for landmark learning in desert ants. Curr. Biol., 20(15):1368-1371.

Vega Vermehren, J. A., Buehlmann, C., Fernandes, A. S. D., and Graham, P. (2020). Multimodal influences on learning walks in desert ants (cataglyphis fortis). J. Comp. Physiol. A Neuroethol. Sens. Neural Behav. Physiol., 206(5):701-709.

Zeil, J. and Fleischmann, P. N. (2019). The learning walks of ants (hymenoptera: Formicidae). Myrmecological News, 29:93110.

Zeil, J., Hofmann, M. I., and Chahl, J. S. (2003). Catchment areas of panoramic snapshots in outdoor scenes. J. Opt. Soc. Am. A Opt. Image Sci. Vis., 20(3):450-469.

# Adversarial Environment Evolution for Robust Pathfinding Neural Cellular Automata 

Sam Earle ${ }^{1,2}$ and Julian Togelius ${ }^{1}$<br>${ }^{1}$ New York University, USA<br>${ }^{2}$ Sakana AI, Japan<br>sam.earle@nyu.edu


#### Abstract

Pathfinding makes up an important sub-component of a broad range of complex tasks in AI, such as robot path planning, transport routing, and game playing. While classical algorithms can efficiently compute shortest paths, neural networks with learned weights could be better suited to adapting these sub-routines to more complex and intractable tasks. Neural Cellular Automata (NCAs) are a particularly promising candidate for learning to solve pathfinding problems given that their purely convolutional architecture amounts to a series of distributed, recursive computations over a grid. This kind of computation is aligned with classical approaches to pathfinding such as Dijkstra's algorithm. NCAs can also be seen as modeling morphogenetic processes, and by this token exhibit robustness against spatially local perturbations and can scale naturally to variable grid-sizes. We train NCAs to solve pathfinding problems using supervised learning and introduce an adversarial training scheme in which data points are mutated during training in order to bias the dataset toward more challengine problems. We find that this approach leads to pathfinding NCAs capable of increased generalization on larger grid-sizes. Our work opens the door to scaling NCAs to more complex problems by automatically evolving curricula during training. ${ }^{1}$


## Motivation

Pathfinding is a crucial sub-routine in many important applications. On a 2D grid, the shortest path problem is useful for robot path planning [Wang et al., 2011] or in transportation routing [Fu et al., 2006]. Long paths (and quantities such as the diameter) are relevant for estimating photovoltaic properties of procedurally-generated microstructures for solar panels [Stenzel et al., 2016, Lee et al., 2021], or the complexity of grid-based video game levels [Khalifa et al., 2020].

Classical algorithms to solve pathfinding and related problems include the Bellman-Ford algorithm [Bellman, 1958, Ford Jr, 1956] for finding the shortest path from a single source node to other nodes, and Breadth-First Search (BFS) [Moore, 1959, Merrill et al., 2012], which models

[^16]the connected nodes using a Dijkstra map [Dijkstra et al., 1959]. Often, these algorithms involve "flooding" values across space (in a potentially distributed manner) in order to measure distance and connectivity.

Neural networks are increasingly used to solve complex problems involving pathfinding subroutines. Therefore, modeling classical pathfinding algorithms in "neurally plausible" ways could be advantageous for holistically solving these more complex problems.

Meanwhile, Neural Cellular Automata (NCAs), which involve recursive, distributed computations over a grid (or graph) structure, have gained a cult following among the Artificial Life community thanks to their simplicity (often involving a relatively minuscule number of learned parameters) and the intuitive, interpretable way in which they model the morphogenesis or "growth" of artifacts across space and time. Such artifacts include images [Mordvintsev et al., 2020], textures [Mordvintsev et al., 2021], warehouses [Zhang et al., 2023], video game levels [Earle et al., 2021], player actions [Variengien et al., 2021] and even neural network controllers [Najarro et al., 2022].

The distributed nature of NCAs make them a natural fit for pathfinding problems. We use such problems as a testbed, and show that NCAs can indeed be trained to solve such problems with high degrees of accuracy. More generally, we ask how we might further improve the robustness of NCAs when solving problems over grids. We propose adversarially evolving environments (here, binary mazes), treating the NCA's loss on a given environment as the environment's fitness. We show that this approach leads to substantially improved performance on held-out test environments including larger grid-sizes, suggesting that online adversarial environment evolution pushes NCAs to learn more robust policies.

## Pathfinding NCAs

Our implementation of Neural Cellular Automata (NCAs) follows that of Mordvintsev et al. [2021]. The NCA consists of one convolutional layer with a $3 \times 3$ kernel and padding of 1, followed by a ReLU activation. In our dataset, mazes are represented as 2D one-hot arrays (with separate chan-

| task | env. evolution | n. hid chan |  | train |  | test |  |  |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|  |  |  |  | $16 \times 16$ |  | $16 \times 16$ |  | $32 \times 32$ |
|  |  |  | n. params | sol. length | pct. complete | accuracies | pct. complete | accuracies |
|  |  |  |  |  |  |  |  |  |
| Shortest path | False | 96 | 86,400 | $9.02 \pm 0.00$ | $78.53 \pm 43.92$ | $77.09 \pm 43.10$ | $76.84 \pm 42.97$ | $65.99 \pm 37.70$ |
|  |  | 128 | 152,064 | $9.02 \pm 0.00$ | $\mathbf{9 9 . 3 7} \pm 0.24$ | $97.44 \pm 0.13$ | $97.61 \pm 0.20$ | $90.06 \pm 2.93$ |
|  | True | 96 | 86,400 | $21.39 \pm 6.99$ | $74.06 \pm 41.44$ | $79.18 \pm 44.26$ | $79.21 \pm 44.28$ | $78.12 \pm 43.67$ |
|  |  | 128 | 152,064 | $\mathbf{2 3 . 7 5} \pm 1.36$ | $92.92 \pm 3.79$ | $\mathbf{9 8 . 8 8} \pm 0.36$ | $\mathbf{9 9 . 3 4} \pm 0.11$ | $\mathbf{9 7 . 1 7} \pm 2.15$ |
| Diameter | False | 96 | 84,672 | $24.09 \pm 0.00$ | $71.58 \pm 40.13$ | $66.15 \pm 36.99$ | $55.66 \pm 31.15$ | $-5.20 \pm 41.28$ |
|  |  | 128 | 149,760 | $24.09 \pm 0.00$ | $\mathbf{7 4 . 0 4} \pm 41.39$ | $67.20 \pm 37.57$ | $58.32 \pm 32.60$ | $27.47 \pm 52.59$ |
|  | True | 96 | 84,672 | $26.06 \pm 1.11$ | $33.94 \pm 19.12$ | $70.47 \pm 39.40$ | $52.90 \pm 29.61$ | $\mathbf{6 0 . 0 1} \pm 33.55$ |
|  |  | 128 | 149,760 | $\mathbf{2 7 . 4 3} \pm 0.15$ | $49.85 \pm 4.07$ | $\mathbf{9 0 . 0 2} \pm 0.44$ | $\mathbf{7 5 . 7 9} \pm 1.57$ | $30.02 \pm 64.80$ |

Table 1: Online adversarial data evolution: Adversarial evolution of mazes during training increases NCAs' generalization ability (particularly to larger $32 \times 32$ grids), as well as the mean shortest-path/diameter length of mazes in the dataset.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-273.jpg?height=234&width=851&top_left_y=970&top_left_x=150)
Figure 1: Learned pathfinding behavior. a) An example maze with shortest path, where blue, green, dark blue, and black respectively represent source, target, path, and wall tiles. b) A learned model computes the shortest path on an out-of-distribution example. It first activates all traversable tiles, then strengthens activations between source and target, while gradually pruning away the rest.

nels for empty tiles, walls, and sources and targets where applicable). These mazes are concatenated with a number of 0-initialized hidden channels, and the NCA is repeatedly applied to these grids 64 times, evolving hidden activations, after which point the first hidden channel is taken as the model's prediction of a path over the grid. Cross-entropy loss is then computed between this prediction and a target path represented as a binary array.

The training dataset comprises $10,00016 \times 16$ mazes, which are generated by randomly placing empty, wall, source, and target tiles until the target is reachable by the source. The resulting paths are relatively short, $\approx 9$ tiles on average. We additionally test models on $32 \times 32$ mazes, which are generated as above, with a mean path length of $\approx 13$ tiles. To generate target paths in the shortest path problem, we compute the shortest path between source and target nodes using BFS. For the diameter problem, we compute shortest paths between all pairs of empty tiles and take the greatest such path as the diameter of the grid.

For the sake of evaluation, accuracy (inverse loss) is normalized against all-zero output, which would achieve $\approx 97 \%$ accuracy on the dataset. Accuracy can thus be negative
when, e.g., a model predicts a path comprising a majority of non-overlapping tiles relative to the true path. Finally, we record whether, after rounding, the output perfectly matches the target path, i.e. the percentage of target paths perfectly completed, denoted as pct. complete in tables. The behavior of a learned pathfinding NCA is pictured in 1 .

## Adversarial Environment Evolution

During training, we evolve grids to maximize a fitness value corresonding to the learned models' loss relative to the target path. When the model's loss falls below a certain threshold, we randomly select among the fittest environments, randomly flip empty/wall tiles with $10 \%$ probability, and (in the shortest path domain) randomly select a new source tile with $5 \%$, and move the target tile to a random empty cell within the connected region occupied by the source tile.

We evaluate the model's performance on these new mazes and use them to replace any less-fit environments in the current training set. In Table 1, we see that such adversarial environment evolution leads to increased generalization on held out test mazes, particularly when these mazes are larger $(32 \times 32)$ than those in the training set ( $16 \times 16$ ). Additionally, this process tends to increase the complexity of the target paths in the training set.

## Conclusion

Neural Cellular Automata can be trained to solve pathfinding problems over grids, modeling solutions via morphogenesis. But generating a balanced dataset with full coverage of the problem space is challenging. We show that this dataset can be evolved during training, adversarially with respect to the learned model, leading to NCAs that generalize better to out-of-distribution environments. Future work may extend our approach to other more complex problems over grids such as puzzle game playing, generating curricula of increasingly complex level layouts and solutions for training robust NCA-based player controllers.

## References

Huijuan Wang, Yuan Yu, and Quanbo Yuan. Application of dijkstra algorithm in robot path-planning. In 2011 Second International Conference on Mechanic Automation and Control Engineering, pages 1067-1069, 2011. doi: 10.1109/MACE.2011.5987118.
L. Fu, D. Sun, and L.R. Rilett. Heuristic shortest path algorithms for transportation applications: State of the art. Computers and Operations Research, 33 (11):3324-3343, 2006. ISSN 0305-0548. doi: https: //doi.org/10.1016/j.cor.2005.03.027. URL https: //www.sciencedirect.com/science/ article/pii/S030505480500122X. Part Special Issue: Operations Research and Data Mining.

Ole Stenzel, Omar Pecho, Lorenz Holzer, Matthias Neumann, and Volker Schmidt. Predicting effective conductivities based on geometric microstructure characteristics. AIChE Journal, 62(5):1834-1843, 2016.

Xian Yeow Lee, Joshua R Waite, Chih-Hsuan Yang, Balaji Sesha Sarath Pokuri, Ameya Joshi, Aditya Balu, Chinmay Hegde, Baskar Ganapathysubramanian, and Soumik Sarkar. Fast inverse design of microstructures via generative invariance networks. Nature Computational Science, 1(3):229-238, 2021.

Ahmed Khalifa, Philip Bontrager, Sam Earle, and Julian Togelius. Pcgrl: Procedural content generation via reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 16, pages 95-101, 2020.

Richard Bellman. On a routing problem. Quarterly of applied mathematics, 16(1):87-90, 1958.

Lester R Ford Jr. Network flow theory. Technical report, Rand Corp Santa Monica Ca, 1956.

Edward F Moore. The shortest path through a maze. In Proc. Int. Symp. Switching Theory, 1959, pages 285292, 1959.

Duane Merrill, Michael Garland, and Andrew Grimshaw. Scalable gpu graph traversal. ACM Sigplan Notices, 47(8):117-128, 2012.

Edsger W Dijkstra et al. A note on two problems in connexion with graphs. Numerische mathematik, 1(1):269271, 1959.

Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklasson, and Michael Levin. Growing neural cellular automata. Distill, 5(2):e23, 2020.

Alexander Mordvintsev, Eyvind Niklasson, and Ettore Randazzo. Texture generation with neural cellular automata. arXiv preprint arXiv:2105.07299, 2021.

Yulun Zhang, Matthew Fontaine, Varun Bhatt, Stefanos Nikolaidis, and Jiaoyang Li. Arbitrarily scalable environment generators via neural cellular automata. Advances in Neural Information Processing Systems, 36: 57212-57225, 2023.

Sam Earle, Justin Snider, Matthew C Fontaine, Stefanos Nikolaidis, and Julian Togelius. Illuminating diverse neural cellular automata for level generation. arXiv preprint arXiv:2109.05489, 2021.

Alexandre Variengien, Stefano Nichele, Tom Glover, and Sidney Pontes-Filho. Towards self-organized control: Using neural cellular automata to robustly control a cart-pole agent. arXiv preprint arXiv:2106.15240, 2021.

Elias Najarro, Shyam Sudhakaran, Claire Glanois, and Sebastian Risi. Hypernca: Growing developmental networks with neural cellular automata. arXiv preprint arXiv:2204.11674, 2022.

# DIRT: The Distributed Intelligent Replicator Toolkit 

Chengrui Wang ${ }^{1,2}$, Chase Van Amburg ${ }^{1}$, Chloe Huangyuan Su ${ }^{1}$, Joseph Bejjani ${ }^{1}$, Yasin Mazloumi ${ }^{1}$, Naeem Khoshnevis ${ }^{1}$, Sham Kakade ${ }^{1}$, Kiante Brantley ${ }^{1}$, Aaron Walsman ${ }^{1}$<br>${ }^{1}$ Kempner Institute, Harvard University, USA<br>${ }^{2}$ Fudan University, China<br>aaronwalsman@fas.harvard.edu


#### Abstract

We introduce DIRT, the Distributed Intelligent Replicator Toolkit, a new evolutionary simulator designed to support GPU-accelerated massive populations and a large physical scale for artificial life research. DIRT is designed to simulate mobile agents in a natural environment that must consume resources in order to survive and reproduce. It uses highly configurable grid world dynamics that include multiple sensor types, complex terrain, water features, and a climate system. Agents in DIRT have traits that control their individual abilities. DIRT is interoperable with a wide variety of policy models and population algorithms. DIRT is built on JAX and as a result, it can support populations of over 10,000 agents on a single GPU. We also provide a rich set of measurement tools and a 3D viewer which allows fine-grained inspection and tracking of individual agents.


## Submission type: Late Breaking Abstract

Data/Code available at: http://github.com/ aaronwalsman/dirt

## Introduction

Artificial Life seeks to understand the emergence of complexity and open-ended evolution in artificial organisms, yet progress has often been constrained by the computational cost of simulating large, interacting populations with sufficiently rich environments. Thus, we present DIRT, a scalable ecological simulator that supports large-scale evolutionary studies on modern accelerators. DIRT is motivated by the long and productive history of grid-world abstractions in ALife and RL, ranging from classical cellular automata and digital evolution to contemporary GPU-native simulators, while pushing scalability, configurability, and inspectability for population-level studies.

Ecological substrate. Grid-based abstractions have a long history in ALife stretching back to cellular automata (Von Neumann et al., 1966; Gardner, 1970; Wolfram and Gad-el Hak, 2003; Adamatzky and Martínez, 2016). DIRT builds on this rich body of work, while adding modern features such as elevation, hydrology and climate dynamics. Environmental processes are modeled as coupled fields,
including discrete water flow with evaporation and precipitation, temperature and moisture cycles influenced by day/night and seasonal rhythms, and resource dynamics spanning light, biomass, energy, and water. Multimodal environmental signals such as visual color maps, odors and diffusion-based chemical cues, audio propagation, thermal measurements, and wind-driven transport, are represented as tensor fields updated via GPU-parallel operators, and then exposed to agents through parameterized sensors. This design builds on ecological simulation approaches in artificial life and multi-agent reinforcement learning (Lu et al., 2024; Sun et al., 2025), but provides a configurable toolkit for a variety of experimental setups.

Agents, traits, and actions. Agents in DIRT are situated in a 2D grid world and must acquire resources to survive and reproduce. This is consistent with the long tradition of digital organisms in ALife (Adami and Brown, 1994; Channon and Damper, 1998; Soros and Stanley, 2014). Each agent is parameterized by a policy that maps the agent's observations to actions and a set of heritable traits that specify the agent's capabilities and costs, echoing earlier work on morphology in evolving virtual creatures (Sims, 1994). Traits determine movement speed, attack radius and strength, metabolic efficiency and others, supporting ecological trade-offs observed in both natural and artificial systems.

DIRT defines a compact but expressive set of discrete action categories including Move, Attack, Eat, Expell, Call, Mark Scent, and Reproduce, which are flexibly modulated by traits and environmental state. This design reflects conventions from earlier agent-based models of flocking and signaling (Reynolds, 1987) and incorporates communicative and reproductive actions inspired by open-ended simulation frameworks such as JaxLife (Lu et al., 2024) and Amorphous Fortress (Charity et al., 2023). Importantly, all actions incur explicit energy and biomass costs, creating resourcebased constraints that enforce trade-offs (e.g., faster movement or stronger attacks consume more energy), in line with ecological and evolutionary principles (MacArthur and Wilson, 2001; Rosenzweig, 1995). In addition, the constants that specify the exact nature of these trade-offs can be spec-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-277.jpg?height=604&width=839&top_left_y=267&top_left_x=155)
Figure 1: Simulator Overview: This map contains $1024 \times 1024$ grid cells and 10000 individual agents, which are too small to see when fully zoomed out. The inset areas progressively zoom in on one patch of terrain to reveal detail.

ified as system-wide parameters.
Interoperable population interface. A key design principle of DIRT is the explicit separation between environmental dynamics and population algorithms. Rather than prescribing heredity rules, learning strategies, or policy representations, DIRT reports demographic events such as births, deaths, and parentage, and delegates the instantiation of new policies and traits to user-provided algorithms. This flexible interface builds on the tradition of open-ended evolutionary systems such as Tierra (Ray, 1992), Avida (Ofria and Wilke, 2004), and Echo (Holland, 1992), while addressing their limitations by decoupling environmental simulation from evolutionary operators.

Conceptually, this modularity parallels the standardized APIs that have accelerated reinforcement learning research, such as Gym (Brockman et al., 2016), PettingZoo (Terry et al., 2021), Gymnasium (Towers et al., 2024), and Gymnax (Lange, 2022), which expose consistent agent-environment interfaces while leaving policy design unrestricted. By not embedding heredity or adaptation mechanisms into the simulator core, DIRT ensures compatibility with diverse research agendas, from open-ended evolution to large-scale multi-agent learning, while maintaining reproducibility and comparability across population-level experiments.

Acceleration and scale. DIRT is implemented entirely in JAX, with all major environment updates, including map field evolution, diffusion-based propagations, hydrology, and sensor extraction, compiled as batched tensor operations for GPUs and TPUs. This minimizes host-device communication overhead, especially if user-defined policies are colocated on the same GPU, which is a known bottleneck in large-scale simulation frameworks (Suarez et al., 2023). This design aligns with recent trends toward acceleratornative environments such as PGX (Koyamada et al., 2023),

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-277.jpg?height=684&width=840&top_left_y=269&top_left_x=1073)
Figure 2: An example of several agent observation and action spaces. In this example, each agent has three motion primitives, shown here as white arrows. Note that these primitives perform different motions for each agent, as they each have different traits. Each agent has a single attack primitive in this example represented by the red box. The agent's visual range is represented by the yellow box. The pixelated background represents the color map that agents see, where blue pixels are water, grey is rock and shades of yellow and brown represent different distributions of resources.

JaxMARL (Rutherford et al., 2023), GigaStep (Lechner et al., 2023), and Craftax (Matthews et al., 2024).

Depending on policy parameterizations, DIRT can sustain populations exceeding 10,000 agents on a single GPU, a scale that enables population-level experimental regimes including ecological niche formation, lineage dynamics, and large-scale selection pressures, which may be difficult to access in smaller systems. This scalability provides a practical foundation for exploring open-ended evolution and emergent population dynamics at scales that were previously unobtainable due to computational constraints.

Measurement and visualization. DIRT incorporates an integrated reporting framework designed to balance scalability with analytical depth. At each timestep, lightweight and customizable summaries are recorded, which can include global environmental fields, population-level statistics, and sampled agent states, while periodic checkpoints capture complete system state to permit exact reconstruction and selective replay. This structure enables both longitudinal analyses of population dynamics and fine-grained investigation of individual behaviors.

Dirt also includes an interactive 3D visualizer that can play back simulated sequences and render different environmental layers alongside agent overlays(Figure 1,2). This interface allows stepwise playback of simulations and targeted reruns from stored checkpoints, thereby supporting detailed inspection of emergent ecological and evolutionary processes.

## References

Adamatzky, A. and Martínez, G. J. (2016). Designing beauty: the art of cellular automata, volume 20 . Springer.

Adami, C. and Brown, C. T. (1994). Evolutionary learning in the 2d artificial life system. In Artificial life IV: proceedings of the fourth international workshop on the synthesis and simulation of living systems, volume 4 , page 377. MIT press.

Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym.

Channon, A. and Damper, R. (1998). Perpetuating evolutionary emergence.

Charity, M., Rajesh, D., Earle, S., and Togelius, J. (2023). Amorphous fortress: Observing emergent behavior in multi-agent fsms. arXiv preprint arXiv:2306.13169.

Gardner, M. (1970). Mathematical games. Scientific american, 222(6):132-140.

Holland, J. H. (1992). Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press.

Koyamada, S., Okano, S., Nishimori, S., Murata, Y., Habara, K., Kita, H., and Ishii, S. (2023). Pgx: Hardwareaccelerated parallel game simulators for reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 45716-45743.

Lange, R. T. (2022). gymnax: A JAX-based reinforcement learning environment library.

Lechner, M., Yin, L., Seyde, T., Wang, T.-H., Xiao, W., Hasani, R., Rountree, J., and Rus, D. (2023). Gigastep one billion steps per second multi-agent reinforcement learning. In Advances in Neural Information Processing Systems.

Lu, C., Beukman, M., Matthews, M., and Foerster, J. (2024). Jaxlife: An open-ended agentic simulator. In ALIFE 2024: Proceedings of the 2024 Artificial Life Conference. MIT Press.

MacArthur, R. H. and Wilson, E. O. (2001). The theory of island biogeography, volume 1. Princeton university press.

Matthews, M., Beukman, M., Ellis, B., Samvelyan, M., Jackson, M., Coward, S., and Foerster, J. (2024). Craftax: A lightning-fast benchmark for open-ended reinforcement learning. In International Conference on Machine Learning (ICML).

Ofria, C. and Wilke, C. O. (2004). Avida: A software platform for research in computational evolutionary biology. Artificial life, 10(2):191-229.

Ray, T. S. (1992). Evolution, ecology and optimization of digital organisms. Santa Fe.

Reynolds, C. W. (1987). Flocks, herds and schools: A distributed behavioral model. In Proceedings of the 14th annual conference on Computer graphics and interactive techniques, pages 25-34.

Rosenzweig, M. L. (1995). Species diversity in space and time. (No Title).

Rutherford, A., Ellis, B., Gallici, M., Cook, J., Lupu, A., Ingvarsson, G., Willi, T., Khan, A., de Witt, C. S., Souly, A., et al. (2023). Jaxmarl: Multi-agent rl environments in jax. arXiv preprint arXiv:2311.10090.

Sims, K. (1994). Evolving 3d morphology and behavior by competition. Artificial life, 1(4):353-372.

Soros, L. and Stanley, K. (2014). Identifying necessary conditions for open-ended evolution through the artificial life world of chromaria. In Artificial Life Conference Proceedings, pages 793-800. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journalsinfo....

Suarez, J., Bloomin, D., Choe, K. W., Li, H. X., Sullivan, R., Kanna, N., Scott, D., Shuman, R., Bradley, H., Castricato, L., et al. (2023). Neural mmo 2.0: A massively multi-task addition to massively multi-agent learning. Advances in Neural Information Processing Systems, 36:50094-50104.

Sun, X., Mangan, M., Peng, J., and Yue, S. (2025). I2bot: an open-source tool for multi-modal and embodied simulation of insect navigation. Journal of the Royal Society Interface, 22(222):20240586.

Terry, J., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sullivan, R., Santos, L. S., Dieffendahl, C., Horsch, C., Perez-Vicente, R., et al. (2021). Pettingzoo: Gym for multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 34:15032-15043.

Towers, M., Kwiatkowski, A., Terry, J., Balis, J. U., De Cola, G., Deleu, T., Goulão, M., Kallinteris, A., Krimmel, M., KG, A., et al. (2024). Gymnasium: A standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032.

Von Neumann, J., Burks, A. W., et al. (1966). Theory of selfreproducing automata. IEEE Transactions on Neural Networks, 5(1):3-14.

Wolfram, S. and Gad-el Hak, M. (2003). A new kind of science. Appl. Mech. Rev., 56(2):B18-B19.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-281.jpg?height=559&width=1747&top_left_y=278&top_left_x=166)
Figure 2: A: Transfer entropy (TE) maps with overlaid vector fields during development (averaged over 1,000 runs). TE first concentrates at the center, later shifting to the edges, with directional flows entering from the sides and propagating vertically. B: TE maps under repeated damage every 180 steps. Red circles mark damaged regions, where TE increases locally and across surrounding areas, indicating distributed coordination during repair.

position of their most similar mature cells to reveal which state developing cells were taking. Figure 1C show developing cells colored with the average position of their top-10 most similar mature cells. Interestingly, this mapping revealed that developing cells with similar colors aggregate to form distinct clusters. The spatial location of these clusters, however, did not necessarily correspond to the mature-stage positions indicated by their colors. Furthermore, these clusters were often adjacent to others with starkly different colors, forming sharp, discrete boundaries between them.

Over the course of development, these clusters undergo a dynamic sequence of appearance, migration, and disappearance. This process can be interpreted as the NCA 'embryo' building intermediate structures and then progressing through their reorganization. Although speculative at this stage, this finding suggests that the developmental process in NCA may exhibit dynamics analogous to the germ layers in biotic ontogeny.

## Information Flows

To characterize how interactions among cells shape developmental dynamics, we estimated information flow using transfer entropy (TE) analysis (Schreiber, 2000). This measure quantifies the directed transfer of information between two time series. A high TE from one entity to another implies that the first entity reduces the uncertainty of the second, thus enabling us to infer a causal relationship.

Figure 2A shows TE maps with overlaid vector fields across developmental stages (averaged over 1,000 runs). Initially, high TE concentrates at the center, then shifts toward the edges as growth proceeds. The vector fields
reveal consistent inflows from the sides and vertical propagation near boundaries, indicating the emergence of directional information flow structures.

We next examined recovery dynamics when damage was introduced after development stabilized. Figure 2B shows TE maps under repeated damage every 180 steps. The first panel (control) reveals that even without perturbation, structured directional TE persists, reflecting ongoing fluctuations in the stable state. In damaged cases (red circles), TE increases around the lesion and spreads to a broader region, suggesting that repair involves coordinated information flow beyond local cells. These findings indicate that information flows in NCA are structured and persistent, shaping development, sustaining stable states, and enabling distributed repair beyond the local site of damage.

## Discussion

The observed developmental dynamics suggests the potential of NCA as a computational model for exploring the principles of biotic-abiotic development through self-organization. Furthermore, the information flows suggest an information theoretic perspective on cell-cell interactions sustaining structural integrity. Future work should aim to further investigate the links between information patterns and the reorganization of these structures, and to establish a formal basis for a general science of development using NCA as a platform.

## Acknowledgements

This work was supported by JSPS KAKENHI Grant Number 24H00707

## References

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing Neural Cellular Automata. Distill.

Schreiber, T. (2000). Measuring Information Transfer. Physical Review Letters, 85(2):461-464.

# Learning navigation on the spatial-social interface 

Patrick Govoni ${ }^{1}$, and Pawel Romanczuk ${ }^{1,2,3}$<br>${ }^{1}$ Institute for Theoretical Biology, Department of Biology, Humboldt Universität zu Berlin, Berlin, Germany<br>${ }^{2}$ Science of Intelligence, Research Cluster of Excellence, Berlin, Germany<br>${ }^{3}$ Bernstein Center for Computational Neuroscience, Berlin, Germany<br>pgovoni21@gmail.com


#### Abstract

Navigation for social organisms rarely is a fully independent activity. Ability as well as spatial density of others in a given environment may critically affect optimal strategy. Individual neural network controlled agents are trained to navigate in different social contexts, where learned social dependence is determined by relative performance and spatial effect, as well as its learned neural representations. Group structure and navigational ability drive phase transitions in social dependence, with behavioral hybridization both along the spatialsocial interface and at differing social distances. Our findings challenge the approach of only inspecting individual behavior for social organisms and highlight the importance of taking a bottom-up approach in understanding how organisms behave.


Many navigational tasks and models focus on individual inferential abilities [ $1,2,3$ ]. Though social organisms, like us, can instead distribute such abilities and decisions across the collective [4,5]. How such inference and behavior develops to coordinate between spatial and social domains has been relatively understudied [6,7,8,9]. Inspired by this gap, we investigate how artificial systems approach this interface, and how it can broaden our understanding of biological behavior.

The task is to navigate to a hidden patch located at a fixed position in a minimal, square environment from random initializations (Fig. 1, bottom right), reflecting the classic Morris Water Maze [1,2]. Each of the four walls are perceivable yet without distinguishing features, as such the four corners are the salient landmarks. We consider a single agent trained in a social context of N other, untrained agents with two possible fixed behaviors and associate navigational skill levels: walking directly to the patch (experienced) and randomly walking (inexperienced). Agent-agent collisions were included to examine social attraction and repulsion with respect to navigating around exploiting agents surrounding the patch [10].

The trained agent visually perceives the environment and other agents by raycast, inspired by [11] and extending the non-social navigation in [12]. Eight rays extend from a single retina, centered at the edge of the agent body, to the first collided object, a boundary wall or an untrained agent. One-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-284.jpg?height=360&width=842&top_left_y=831&top_left_x=1073)
Figure 1: Agent flow \& train-test methodology. Left: (clockwise from bottom left) visual encoding, information processing, action conversion, environment update. Right: training with varying group structure and testing with a point perturbation.

hot encoded collisions identify which objects are at which angles with respect to the agent. Identifying other agents is defined along action-location status: whether the agent is on the patch and exploiting, or off the patch and exploring.

The information processing flow consists of a convolutional neural network (CNN), single perceptron layer, and a linear output layer, mirroring previous work [12]. Network output bottlenecks into a single action that updates agent location and orientation for the next timestep, constrained as a linear ratio between turning angle and speed, describing a need to slow down in order to turn.

Agent performance is calculated as the time taken to reach the patch, plus the remaining distance if the agent has not reached the patch within the simulation time limit. An evolutionary strategies (ES) algorithm optimized the network in pursuit of this goal.

Agents trained in this environment affords at least two potential learned behaviors: an individual navigation strategy that makes decisions relative to spatially fixed landmarks, or a social strategy that consists of following or avoiding the other agents. The ratio between direct and random untrained agents reflects the environmental social skill available, while the total number reflects social density. The social skill ratio was hypothesized to negotiate the degree to which training

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-285.jpg?height=586&width=840&top_left_y=269&top_left_x=150)
Figure 2: Directional divergence \& neural activation. Left: movement trajectories for No-Social (NS) \& BeaconExploiter (BEt), revealing relative performance (green) \& spatial (red) effects, with the latter quantified as directional divergence. Right: example neurons in original training (OG) \& test environments.

leads to a dependency on social information, despite having the possibility of learning an individual spatial navigation strategy.

In order to distinguish a learned dependency on social relative to spatial landmark information, trained agents were tested in two separate environments (Fig. 1): one without any other agents (No-Social, NS), and one with an exploiting agent at the patch center (Beacon-Exploiter, BEt). Relating navigation performance between these two test environments establishes a measure for social dependence, where a positive value signifies social information is beneficial.

Navigational performance alone, however, was found to not capture the degree to which social information is used spatially across the environment. Those with low performance difference can differ markedly in the degree to which the trajectories are affected by other agents (Fig. 2i/ii), likewise relying on others to reach the patch may not correlate with spatial dependence (Fig. 2iii). Building on an entropic measure from previous work [12], directional divergence uses Jensen-Shannon divergence on spatially binned distributions of agent orientations. If a bin has similar distributions of agent orientations in each test environment, the directional divergence is low, which corresponds to small impact on trajectories in this location, i.e. exhibits a low relative spatial effect.

Varying the social skill ratio was found to coordinate a discontinuous phase transition (Fig. 3). Performance difference steps up for training environments with two direct agents, with an accompanying phenomenon describing the change in directional divergence past three direct agents. Variance for the stair plot data, not shown, spikes at two

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-285.jpg?height=586&width=670&top_left_y=269&top_left_x=1160)
Figure 3: Relative performance \& spatial effects. Stair plots report medians of 40 repeats.

to three direct agents, further suggesting a phase transition. Though the spatial effect appears to correlate with performance difference at low density social contexts, the two decorrelate with greater numbers of direct agents, potentially due to collisions at the patch boundary.

Hybridization between individual and social navigation is also possible for the majority of agents that prefer straight trajectories. For both agents in Fig. 2iii and iv, a zone demarcated by elliptical manifolds specified by visual angles, as analyzed in the preceding study [12], governs such transitions where the agent is able to self-navigate outside, but must depend on others for the rest of the way. This appears to describe observed interplay between different navigational processes, where an interface may depend on relative information value but also individual experience. More work is needed to analyze such interfaces and relate it to empirical studies.

Social navigation may be similarly demarcated with respect to distance from another agent. Given there are multiple other agents in the environment, at far distance it may be best to learn a light attraction, while a stronger attraction for mid-range (Fig. 2v). Though at very close ranges, it would be best to learn a soft repulsion to avoid costly collisions. These properties have been defined as assumptions elsewhere for flocking and schooling simulations in the field of collective behavior [11, 13], however here they have been entirely learned, are for a navigational rather than a swarming task, and use direct perceptual inputs rather than rely on phenomenological idealizations [6, 11, 14, 10].

Acknowledgements: P.G. acknowledges support by the Elsa-Neumann-Stipendium by the state of Berlin under Nachwuchsförderungsgesetz (NaFöG) - application number H75014. P.R. acknowledges funding by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy - EXC 2002/1 "Science of Intelligence" - project number 390523135.

## References

Eloy Parra-Barrero, Sandhiya Vijayabaskaran, Eddie Seabrook, Laurenz Wiskott, and Sen Cheng. A map of spatial navigation for neuroscience. Neuroscience \& Biobehavioral Reviews, 152:105200, September 2023.

Andrea Banino, Caswell Barry, Benigno Uria, Charles Blundell, Timothy Lillicrap, Piotr Mirowski, Alexander Pritzel, Martin J. Chadwick, Thomas Degris, Joseph Modayil, Greg Wayne, Hubert Soyer, Fabio Viola, Brian Zhang, Ross Goroshin, Neil Rabinowitz, Razvan Pascanu, Charlie Beattie, Stig Petersen, Amir Sadik, Stephen Gaffney, Helen King, Koray Kavukcuoglu, Demis Hassabis, Raia Hadsell, and Dharshan Kumaran. Vector-based navigation using grid-like representations in artificial agents. Nature, 557(7705):429433, May 2018.

Stefano Recanatesi, Matthew Farrell, Guillaume Lajoie, Sophie Deneve, Mattia Rigotti, and Eric Shea-Brown. Predictive learning as a network mechanism for extracting low-dimensional latent space representations. Nature Communications, 12(1):1417, March 2021.

Thomas Mueller, William F. Fagan, and Volker Grimm. Integrating individual search and navigation behaviors in mechanistic movement models. Theoretical Ecology, 4(3):341-355, August 2011.

Jacob D. Davidson, Matthew M. G. Sosna, Colin R. Twomey, Vivek H. Sridhar, Simon P. Leblanc, and Iain D. Couzin. Collective detection based on visual information in animal groups. Journal of the Royal Society Interface, July 2021.

Ariana Strandburg-Peshkin, Colin R. Twomey, Nikolai W. F. Bode, Albert B. Kao, Yael Katz, Christos C. Ioannou, Sara B. Rosenthal, Colin J. Torney, Hai Shan Wu, Simon A. Levin, and Iain D. Couzin. Visual sensory networks and effective information transfer in animal groups. Current Biology, 23(17):R709-R711, September 2013.

Matthew Schafer and Daniela Schiller. Navigating Social Space. Neuron, 100(2):476-489, October 2018.

Isabella C. Wagner, Luise P. Graichen, Boryana Todorova, Andre Lüttig, David B. Omer, Matthias Stangl, and Claus Lamm. Entorhinal grid-like codes and time-locked network dynamics track others navigating through space. Nature Communications, 14(1):231, January 2023.

Angelo Forli and Michael M. Yartsev. Hippocampal representation during collective spatial behaviour in bats. Nature, 621(7980):796-803, September 2023.

David Mezey, Dominik Deffner, Ralf HJM Kurvers, and Pawel Romanczuk. Visual social information use in collective foraging, December 2023.

Renaud Bastien and Pawel Romanczuk. A model of collective behavior based purely on vision. Science Advances, 6(6), February 2020.

Patrick Govoni and Pawel Romanczuk. Visuospatial navigation without distance, prediction, or maps, November 2024.

Pawel Romanczuk and Lutz Schimansky-Geier. Swarming and pattern formation due to selective attraction and repulsion. Interface Focus, 2(6):746-756, December 2012.

Winnie Poel, Claudia Winklmayr, and Pawel Romanczuk. Spatial structure and information transfer in visual networks, June 2021.

# Self-Organising Digital Circuits 

Marcello Barylli ${ }^{1}$, Gabriel Béna ${ }^{2}$, Alexander Mordvintsev ${ }^{3 *}$, Eleni Nisioti ${ }^{1}$ and Sebastian Risi ${ }^{1}$<br>${ }^{1}$ IT University of Copenhagen<br>${ }^{2}$ Imperial College London<br>${ }^{3}$ Google, Paradigms of Intelligence Team<br>bary@itu.dk, g.bena21@imperial.ac.uk


#### Abstract

Fault tolerance in computing has traditionally relied on hardware redundancy and error-correcting codes. Biological intelligence, in contrast, exhibits mechanisms that allow for adaptive resilience. Natural automata like the brain maintain function by re-organising around damaged components. Inspired by this principle, we introduce an architecture for self-organising, fault-tolerant digital circuits. Our hierarchical architecture uses a Graph Transformer (GT) that metalearns to configure the lookup tables (LUTs) of the circuits. Unlike Neural Cellular Automata (NCAs), which learn to directly generate a target, our network learns an indirect, functional policy. This allows it to reconfigure permanently damaged or topologically randomised circuits, rather than relying on regeneration. We demonstrate that our meta-learned network can rapidly reconfigure circuits with previously unseen damage patterns. Our work brings the principles of selforganising systems to the practical domain of digital hardware, where distributed models can offer benefits like compactness and robustness against noise.


## Submission type: Late Breaking Abstract

Data/Code + Circuit Visualization available here

## Introduction

Engineered circuits typically achieve fault tolerance through static redundancy, whereas biological systems exhibit graceful degradation through dynamic re-organisation around damage (Nudo, 2013). Inspired by von Neumann's principle of systems that "operate across errors" (von Neumann, 1966), we develop a framework for self-organising digital circuits.

We adapt principles from Neural Cellular Automata (NCAs) and graph variants Mordvintsev et al. (2020); Grattarola et al. (2021), where cell states typically form the output pattern directly. In our approach, these states are the entries of Boolean Lookup Tables (LUTs) that collectively perform a given computation. The update rules are trained only on the final circuit output (eg binary multiplication of input bits), not on specific LUT target values. This induces an

[^17]indirect, functional relationship between the self-organising process and the target.

Our work provides a concrete, hardware-adjacent model for ALIFE principles, pushing Graph NCA-like selforganising systems closer to solving practical engineering problems. Its adaptive capacity relies on forward passes only, potentially allowing for on-chip deployment in reprogrammable hardware, such as Field Programmable Gate Arrays (FPGAs) (Carter et al., 1986). This stands in contrast with backpropagation, which requires circuit components to be differentiable Sunada et al. (2025).

## Methods

We use a differentiable circuit representation where each gate is a LUT. Following Petersen et al. (2022), a continuous relaxation of the LUTs enables gradient-based optimisation. Unlike their approach, we use recursive multilinear interpolation to define gate identity, rather than a learned categorical distribution.

We represent each circuit as a graph where nodes correspond to gates and edges to wires. Each node holds the gate's LUT parameters, a hidden state, and positional encodings. This graph structure, defined by the circuit's wiring, constrains the bidirectional message passing in our Graph Transformer (GT) (Yuan et al., 2025). The GT acts as the meta-learner, trained on a persistent pool of circuit instances representing the distribution of optimisation examples (Fig. $1)$.

Inner Loop (Circuit "Growth" and Reconfiguration): For each circuit in a batch sampled from the pool, the GT performs a forward pass of $N$ message passing steps to compute updates for each node's LUT parameters, analogous to the NCA growth and regeneration steps.

Outer Loop (Meta-Parameter Update): The updated circuits are executed, their outputs are compared against a target Boolean function, and a meta-loss is calculated. This gradient is used to GT's weights, improving its general policy for reconfiguring circuits.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-289.jpg?height=792&width=680&top_left_y=264&top_left_x=230)
Figure 1: Summary of training framework. 1. Sampling of circuits (with possibly multiple damage or wiring patterns) from the pool. 2. Inner loop: optimization using the GT for message passing between LUTs, updating states. 3. Outer Loop: Functional loss $\mathcal{L}$ is computed by running final optimized circuit. We then backpropagate through the entire inner loop to update the GT's parameters.

## Experiments

Wire Shuffles. To illustrate the relationship between structure and function, we implement wire shuffle experiments. The goal is to maintain input-output mapping (functional invariance) across variations of the circuit's wiring topology. Preliminary results show that a single GT optimiser is able to optimise for arbitrary wiring on fixed topologies (number of gates and layers), in the case of simple Boolean task such as bit-reverse. Extension to more complicated tasks such as binary multiplication, as well as achieving proper scale-free optimisation (single policy for any topologies), are still under way.

Stuck-at Fault Gate Failures. Another test of functional invariance is component failure. We simulate failure via "stuck-at 0 " faults, where a damaged gate's output is permanently clamped to zero and the gate is removed from the message-passing graph on the GT level. This captures a realworld scenario (Mei, 1974) and forces the system to develop a robust strategy for reconfiguring remaining intact gates.

Figure 2 shows such a reconfiguration trajectory. First, a circuit is configured via backpropagation to solve a binary multiplication task. 20 gates are damaged randomly, and the GT performs the reconfiguration.

Error bars indicate performance across 256 damage patterns. On both seen and unseen tasks, the meta-learner is

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-289.jpg?height=551&width=833&top_left_y=274&top_left_x=1080)
Figure 2: Reconfiguration Trajectory following damage. Blue shaded region indicates pre-damage timesteps. The orange dashed line indicates damage time point. Green indicates recovery trajectory timesteps.

capable of recovering a significant but not perfect performance, showcasing graceful degradation (Zhou et al., 2007).

To compare the GT's message passing scheme with recovery via backpropagation, we measure the Hamming distance of the reconfigured circuits for both methods to see which one is further from the unperturbed baseline. The Hamming distance measures in how many positions (ie circuit-wide LUT entries) the reconfigured circuit differs from the baseline in hard mode (Hamming, 1950). Figure 3 shows that the GT achieves comparable accuracy with a lower Hamming distance across 256 damage patterns. Significance testing remains to be done.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-289.jpg?height=304&width=830&top_left_y=1631&top_left_x=1080)
Figure 3: Measuring Perturbation Response via Hamming Distance. For remaining undamaged gates, fewer LUT entries are flipped as a reconfiguration following damage for GT than for Backpropagation.

## Work in Progress

Explore out-of-distribution generalisation (larger damage sizes, different circuits topologies). Measure response to local perturbations in more detail, which requires visualisations of the circuits themselves. Visualise the reconfiguration of remaining gates to see whether the GT exhibits a more localised response than backpropagation.

## Acknowledgments

Funded by the European Union (ERC, GROW-AI, 101045094). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council.

## References

Carter, W., Duong, I., Freman, R., Hsieh, H., Ja, J., Mahoney, J., Ngo, N., and Sac, S. L. (1986). A user programmable reconfigurable logic array.

Grattarola, D., Livi, L., and Alippi, C. (2021). Learning Graph Cellular Automata. arXiv:2110.14237 [cs].

Hamming, R. W. (1950). Error detecting and error correcting codes. The Bell System Technical Journal, 29(2):147-160.

Mei, K. (1974). Bridging and Stuck-At Faults. IEEE Transactions on Computers, C-23(7):720-727.

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing Neural Cellular Automata. Distill, 5(2):e23.

Nudo, R. J. (2013). Recovery after brain injury: mechanisms and principles. Frontiers in Human Neuroscience, 7:887.

Petersen, F., Borgelt, C., Kuehne, H., and Deussen, O. (2022). Deep Differentiable Logic Gate Networks. arXiv:2210.08277 [cs].

Sunada, S., Niiyama, T., Kanno, K., Nogami, R., Röhm, A., Awano, T., and Uchida, A. (2025). Blending Optimal Control and Biologically Plausible Learning for NoiseRobust Physical Neural Networks. Physical Review Letters, 134(1):017301. Publisher: American Physical Society.
von Neumann, J. (1966). Theory of Self-Reproducing Automata. University of Illinois Press.

Yuan, C., Zhao, K., Kuruoglu, E. E., Wang, L., Xu, T., Huang, W., Zhao, D., Cheng, H., and Rong, Y. (2025). A Survey of Graph Transformers: Architectures, Theories and Applications. arXiv:2502.16533 [cs].

Zhou, L., Prabhakaran, V., Ramasubramanian, V., Levin, R., and Thekkath, C. A. (2007). Graceful degradation via versions: specifications and implementations. In Proceedings of the twenty-sixth annual ACM symposium on Principles of distributed computing, PODC '07, pages 264-273, New York, NY, USA. Association for Computing Machinery.

# Scaffolding development via morphogenetic gradients 

Milton L. Montero ${ }^{1}$, Elias Najarro ${ }^{1}$, Eleni Nisioti ${ }^{1}$ and Sebastian Risi ${ }^{1,2}$<br>${ }^{1}$ IT University of Copenhagen, Denmark; ${ }^{2}$ Sakana AI, Japan<br>mlle@itu.dk


#### Abstract

Neural Cellular Automata have enabled the creation of artificial self-organising systems of a hitherto unfeasible scale and complexity. Traditionally, the main hurdle faced by NCAs is figuring out how to effectively break symmetries amongst a priori identical cells. To overcome this issue, several approaches have been proposed such as stochasticity in the update rules and randomizing the initial patterns. Taking inspiration from developmental biology, we explore an alternative method using morphogenetic gradients: variations of the concentration of signalling molecules along the developmental substrate. Said gradients serve a crucial role during biological development, providing positional information, establishing boundaries between regions and generally serving as a scaffold that helps determine cell fate. We show that by incorporating this mechanism into NCAs, we are able to facilitate the symmetry breaking process, enabling more robust and faster training and development. Future work will explore how to use these gradients to manipulate the developmental process in predictable ways such as altering the scale, orientation and even the repetition of common structures in the patterns generated by the NCAs.


## Submission type: Late Breaking Abstracts

Code: https://github.com/miltonllera/ morphogen-dev

## Introduction

Developmental systems in biology rely on a wide variety of mechanisms in order to robustly produce their target morphology. One such mechanism is the use of morphogenetic gradients [6]. These non-uniform patterns of biochemical signal concentrations help establish body axes during early development, serve as positional information which help determine cell fate, and function as guides during cell migration, amongst other uses $[14,5,1,8]$

Neural Cellular Automata (NCAs) have recently gained popularity as a way to implement models of selforganisation [7]. The main benefit of the approach is the ability to learn the rules that guide the system from data. This not only frees researchers from having to come up with said rules themselves, which are not always trivial, but it also

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-292.jpg?height=266&width=682&top_left_y=826&top_left_x=1153)
Figure 1: Morphogenetic gradients. (Left) Difference in the concentrations of morphogens help determine cell fate. (Right) We augment NCAs with various, fixed 2D patterns.

enables the system to scale up to more complex patterns than the ones that were previously possible.

This qualitative leap in capabilities comes at a cost. Breaking symmetries among cells in the automaton becomes a critical issue; if left unchecked, the system tends to converge on degraded patterns [7]. Moreover, training is now required, and the system's distributed, recurrent nature makes it especially susceptible to instabilities. Finally, we would like to be able to control properties of the system such as orientation, scale and even shape without having to modify the parameters of the NCA. This would produce more general models that can more flexibly traverse the morphological space and thus generalise better [8].

In this work we explore how the use of morphogenetic gradients, understood as a fixed underlying scaffold which locally conditions the NCA update, can help to address these issues. We take inspiration from several studies in developmental biology which have show how morphogenetic gradients serve as scaffolds for development [8, 9, 2, 12].

## Methods

We implemented this idea by augmenting the hidden channels of the NCA with fixed 2D-patterns such as the ones seen in Figure 1. These provide different information to the cells such as their distance from the centre (as with the Gaussian pattern in the top row) or their position relative to the vertical and horizontal axes (as in the bottom row) of development. The functioning of the NCAs proceeds as in [7], except that morphogenetic information is appended to the

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-293.jpg?height=243&width=846&top_left_y=274&top_left_x=155)
Figure 2: Morphogen Neural Cellular Automaton (mNCA) architecture. The target shape is represented by three visible RGB channels. The transition function of the NCA proceeds as follows: first information spreads via fixed kernels. Second, the morphogen fields are concatenated to the resulting perception vector at each cell. Finally they are updated by a fully connected neural network.

output of the perception function, serving as extra input to the update function (see Figure 2).

We compare this model (called mNCA) against the standard Growing NCA [7], and and Noise NCA which breaks symmetries by initializing the substrate with a uniform random pattern without growth and has a deterministic update rule [10]. We train all models five times on twowsingle emojis (smiley and salamander) and two shapes (cross and square) using a goal-guided approach [13]. Training ran for 15,000 iterations an we used the Adam optimizer with a learning rate of $1 e-3$ [3] and no other regionalisations techniques (e.g. learning rate schedulers or intermediate state pools).

## Results

The results for single targets are summarized in Table 1. Across all benchmarks, the mNCA achieved a lower training error (measures using Mean Squared Error), even surpassing the NoiseNCA, which does not have to grow the shape. Furthermore, after training it is able to converge much quicker to the target pattern (as measured by reaching a validation error below 1.0 MSE ), usually under 40 to 50 steps.

While the above already illustrates the potential benefits of using morphogens, it could be argue that such information makes the self-organization task too simple, since each position always maps to one specific value. However, we

| Dataset | Model | MSE | Steps-to-conv |
| :--- | :--- | :--- | :--- |
| Shapes | NCA | $0.3 \pm 0.2$ | $69.6 \pm 7.32$ |
|  | Noise NCA | $2.0 \pm 4.0$ | $65.4 \pm 13.45$ |
|  | mNCA | $\mathbf{0 . 0 5} \pm 0.02$ | $\mathbf{4 7} \pm 0.90$ |
| Emojis | NCA | $4.56 \pm 1.27$ | $78.48 \pm 9.2$ |
|  | Noise NCA | $4.64 \pm 2.26$ | $61.5 \pm 4.15$ |
|  | mNCA | $\mathbf{1 . 1 5} \pm 0.5$ | $\mathbf{5 9} \pm 3.9$ |

Table 1: MSE and average steps to convergence during development

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-293.jpg?height=359&width=586&top_left_y=274&top_left_x=1199)
Figure 3: Learning curves when trained on multiple targets. Mean squared error ( y -axis) as a function of training step (x-axis) for the three different models on when trained on all emojis. We only show the first 5000 iterations to illustrate the faster convergence of the mNCA variant.

also observed similar results when training the models to produce several targets when using a shape-dependent conditioning vector. In Figure 3 we show the learning curves for goal-guided variants of the models trained to reproduce 10 emojis. We observed that the mNCA was capable of more efficient scaling to this much harder task. Thus this provides evidence that morphogens could server as a general mechanism that scaffolds development across multiple targets which would otherwise quickly overwhelm the NCAs capacity. From a more machine learning point of view, using morphogens also lead to more stable training. Indeed, the mNCA model showed lower variance in its training curve and rarely falls into a common local minima where the parameters of the update rule leads all cells to die at the start of development.

## Discussion

The results in this paper demonstrate the computational advantages of morphogenetic information, as they provide a scaffold on which developmental processes can unfold with increased robustness across both single and multiple targets. The current study has focused on the benefits for training NCA models, and the effect on development after said training is complete. The next steps will explore how morphogenetic information can enable the manipulatio of the development process in more systematic ways [15]. For example, it would be desirable to manipulate the orientation, scale, and even shape of the patters [11, 4], or the number and arrangement of different modules [2]. This would open the door to more flexible cellular automata systems which can not only solve more complex pattern generation task [11], but also serve as potential models for sophisticated phenomena in developmental biology [15]

## Acknowledgements

This project was supported by the European Research Council (ERC) grant (GA no. 101045094, project GROWAI) and by the Novo Nordisk Foundation, grant number NNF23OC0086722.

## References

J. B. A. Green and J. Sharpe. Positional information and reactiondiffusion: two big ideas in developmental biology combine. 142(7):1203-1211. ISSN 0950-1991. doi: 10.1242/ dev.114991. URL https://doi.org/10.1242/dev. 114991.
M. Khona, S. Chandra, and I. Fiete. Global modules robustly emerge from local interactions and smooth gradients. 640(8057):155-164. ISSN 1476-4687. doi: 10.1038/ s41586-024-08541-3. URL https://www.nature. com/articles/s41586-024-08541-3. Publisher: Nature Publishing Group.
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. URL http://arxiv.org/abs/1412.6980.
A. Kumar, J. Clune, J. Lehman, and K. O. Stanley. Questioning representational optimism in deep learning: The fractured entangled representation hypothesis. 2025. URL https: //arxiv.org/abs/2505.11581.
A. D. Lander. Morpheus unbound: Reimagining the morphogen gradient. 128(2):245-256. ISSN 00928674. doi: 10.1016/j.cell.2007.01.004. URL https://www.sciencedirect.com/science/ article/pii/S0092867407000517.
H. Meinhardt. Models of biological pattern formation: From elementary steps to the organization of embryonic axes. In S. Schnell, P. K. Maini, S. A. Newman, and T. J. Newman, editors, Current Topics in Developmental Biology, volume 81 of Multiscale Modeling of Developmental Systems, pages 1-63. Academic Press. doi: 10.1016/S0070-2153(07) 81001-5. URL https://www.sciencedirect.com/ science/article/pii/S0070215307810015.
A. Mordvintsev, E. Randazzo, E. Niklasson, and M. Levin. Growing neural cellular automata. 5(2):e23. ISSN 2476-0757. doi: 10.23915/distill.00023. URL https://distill.pub/ 2020/growing-ca.
L. S. Mosby, A. E. Bowen, and Z. Hadjivasiliou. Morphogens in the evolution of size, shape and patterning. 151(18): dev202412. ISSN 0950-1991. doi: 10.1242/dev. 202412. URL https://doi.org/10.1242/dev. 202412.
I. N. Nuñez, T. F. Matute, I. D. Del Valle, A. Kan, A. Choksi, D. Endy, J. Haseloff, T. J. Rudge, and F. Federici. Artificial symmetry-breaking for morphogenetic engineering bacterial colonies. 6(2):256-265. doi: 10 . 1021/acssynbio.6b00149. URL https://doi.org/10. 1021/acssynbio.6b00149. Publisher: American Chemical Society.
E. Pajouheshgar, Y. Xu, and S. Süsstrunk. NoiseNCA: Noisy seed improves spatio-temporal continuity of neural cellular automata. URL http://arxiv.org/abs/2404.06279.
E. Randazzo, A. Mordvintsev, and C. Fouts. Growing steerable neural cellular automata. URL http://arxiv.org/ abs/2302.10197.
J. Raspopovic, L. Marcon, L. Russo, and J. Sharpe. Digit patterning is controlled by a bmp-sox9-wnt turing network modulated by morphogen gradients. 345(6196):566-570. doi: 10.1126/ science.1252960. URL https://www.science.org/
doi/10.1126/science.1252960. Publisher: American Association for the Advancement of Science.
S. Sudhakaran, E. Najarro, and S. Risi. Goal-guided neural cellular automata: Learning to control self-organising systems. In From Cells to Societies: Collective Learning across Scales, 2022. URL https://arxiv.org/abs/2205.06806.
L. Wolpert. Positional information and pattern formation. 117: 597-608. ISSN 1557-8933. doi: 10.1016/bs.ctdb.2015.11. 008.
I. Zarkesh, M. Kazemi Ashtiani, Z. Shiri, S. Aran, T. Braun, and H. Baharvand. Synthetic developmental biology: Engineering approaches to guide multicellular organization. 17(4): 715-733. ISSN 2213-6711. doi: 10.1016/j.stemcr.2022. 02.004. URL https://www.sciencedirect.com/ science/article/pii/S2213671122000935.

# Evolution as an alternative to Reinforcement Learning for Continual Control 

Eleni Nisioti, Kathrin Korte, Sebastian Risi<br>IT University of Copenhagen<br>enis@itu.dk


#### Abstract

Learning under environmental changes has remained a central challenge in neural network-based optimization for decades. In artificial evolution, on the other hand, change is not merely a concern but a driver of adaptation. Here, we evaluate various neuro-evolution algorithms on standard continual control tasks and show that they significantly outperform reinforcement learning. We propose that neuroeevolution can be a promising approach towards tackling continual adaptation in the deep learning era.


Code: https://github.com/eleninisioti/continual_control Submission type: Late Breaking Abstract

## Introduction

Continual learning is the constant development of increasingly complex behaviors (Khetarpal et al., 2022). Biological adaptive organisms exhibit it to various extents, with some species adjusting to seasonal variations, some transferring skills across increasingly complex tasks and others exhibiting a remarkably ability to switch among widely diverse behaviors within minutes (Shelton and Martins, 2017). Artificial agents, on the other hand, are classically trained in stationary environments and often fail catastrophically when encountering novel situations (Farebrother et al., 2020). Failure can come in the form of either catastrophic forgetting, where the environmental change harms previously acquired skills or plasticity loss, where the change does not induce the necessary behavioral change. These challenges have been studied in both supervised and reinforcement learning tasks, with approaches often focusing on improving the optimisation objective, the training procedure or the architecture of the artificial neural networks (Khetarpal et al., 2022; Wang et al., 2024; Mirzadeh et al., 2022).

Evolutionary optimization is a rich family of gradientfree, population-based algorithms that have recently been shown to perform on par or even outperform deep reinforcement learning in control tasks when evolving the weights of neural networks (Such et al., 2018; Salimans et al., 2017;

Chalumeau et al., 2023; Nisioti et al., 2025). From its inception to today, this field has often found inspiration in biological evolution to design mechanisms and propose hypotheses (Koza, 1994; Holland, 1992; Stanley and Miikkulainen, 2002; Chan, 2023). For this reason, continual complexification, referred to also as open-endedness, has been central in its development and not just a desiderata arising as an afterthought (Langton, 1997; Soros and Stanley, 2014; Soros et al., 2017). Studies of artificial evolution have, therefore, often focused on how environmental variability can affect evolutionary dynamics, showing that some changes can promote the emergence of adaptation mechanisms (Nisioti and Moulin-Frier, 2022) or accelerate the pace of evolution (Lehman and Miikkulainen, 2015). This raises a question: can evolutionary optimization handle the continual learning challenges that researchers are trying to address by appropriately engineering reinforcement learning algorithms?

To answer this question we here evaluate different evolutionary algorithms in a continual learning set-up, where the environment changes periodically across evolution. Differently from a previous study that considered tasks complexifying based on a curriculum (Nisioti et al., 2025), we here focus on a well-established set of continual learning tasks, where changes do not affect the complexity of the task but shift the observations the agent receives. Such a set-up captures environmental distribution shifts that can offer occur when an artificial system is deployed to monitor the real world. We consider both genetic algorithms and evolutionary strategies and show that they all handle environmental changes significantly better than RL, with the genetic algorithm performing best across tasks.

## Study set-up

Tasks We consider a simple variation of three classical control tasks with discrete actions, continuous observations and episode length of 200 (Muppidi et al., 2024): a) in Acrobot the agent controls the actuator at the joint of a two-link chain, fixed at one end, to swing the free end above a target height. The highest score possible is -60 and corresponds to reaching the target height within 60 steps in the environ-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-297.jpg?height=764&width=1374&top_left_y=271&top_left_x=335)
Figure 1: A) Overall comparison in continual control We consider three simple control tasks where random noise is added to the observations and is being resampled every 200 generations (grey vertical lines indicate the moment at which the noise changes) B) Statistical comparison of cumulative rewards in continual learning setting We compare the cumulative rewards collected by the different methods. The number of stars indicates the level of significance C) Comparison without environmental changes

ment b) in Cartpole the agent is controlling a pole attached on a cart. The optimal reward is 200 , which corresponds to keeping the pole up for 200 steps c) in Mountain Car the agent is controlling a simple car that needs to escape a deep valley. The agent receives a penalty for each timestep and a high reward once it escapes the valley, making this a challenging spare reward problem. The optimal reward is -100 . To adapt these tasks to a continual learning set-up we add a noise vector to the observations whose values change every 200 generations. This vector is sampled from a normal distribution with a standard deviation of 2 . We run the simulation for 10 changes.

Algorithms We evaluate both population-based (SimpleGA) and distribution-based evolutionary approaches (CMA-ES, OpenES) implemented in evosax(Lange, 2022). The population size is 512 and we have tuned their hyperparameters in the standard version of the MountainCar (provided in the online repo). We also evaluate Proximal Policy Optimisation (PPO), a state-of-the-art reinforcement learning algorithm (Muppidi et al., 2024; Ahmed et al., 2020). To ensure that all algorithms are compared for the same sample complexity, we change the environment for PPO every $200 * 200 * 512$ steps and train for 10 changes.

## Results

In Figure 1(A) we compare the performance of the different methods. We observe that, in all tasks, PPOs fails catas-
trophically and evolution manages to recover in most cases, with the genetic algorithm performing best. As some tasks may be challenging for some methods independently of task changes, we also present performance in the classical form of these tasks in Figure 1(C). Here we observe that GA and OpenES converge slowly ib MountainCar (they require about 800 generations), which indicates that they did not reach optimal performance in the continual setting due to slow convergence rather than the added challenge of noise. To quantify differences in performances we calculate the rewards that methods accumulated across the whole simulation and perform pairwise statistical comparisons using the Mann-Whitney U test with Bonferonni corrections.

## Discussion

Change, seen as an obstacle that we need to design our algorithms around in classical deep learning, is recognized as an important driver of adaptability in both biological (Maslin et al., 2015; Ellefsen, 2014) and artificial evolution (Lehman and Miikkulainen, 2015; Soros and Stanley, 2014). Here we show that evolutionary algorithms significantly outperform reinforcement learning in continual learning tasks paving the way for using evolution as an alternative to developing novel RL algorithms. We plan to extend our study to more environments, evolutionary algorithms and RL variants designed for continual learning to further showcase this potential. We will also investigate whether the genetic algorithm performs best due to having more diversity compared to distribution-
based evolution.

## Acknowledgements

Funded by the European Union (ERC, GROW-AI, 101045094). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council.

## References

Ahmed, O., Träuble, F., Goyal, A., Neitz, A., Bengio, Y., Schölkopf, B., Wüthrich, M., and Bauer, S. (2020). CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning. arXiv:2010.04296 [cs].

Chalumeau, F., Boige, R., Lim, B., Macé, V., Allard, M., Flajolet, A., Cully, A., and Pierrot, T. (2023). Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discovery. arXiv:2210.03516 [cs].

Chan, B. W.-C. (2023). Towards Large-Scale Simulations of OpenEnded Evolution in Continuous Cellular Automata. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation, GECCO '23 Companion, pages 127130, New York, NY, USA. Association for Computing Machinery.

Ellefsen, K. O. (2014). The Evolution of Learning Under Environmental Variability. pages 649-656.

Farebrother, J., Machado, M. C., and Bowling, M. (2020). Generalization and Regularization in DQN. arXiv:1810.00123 [cs].

Holland, J. H. (1992). Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. The MIT Press.

Khetarpal, K., Riemer, M., Rish, I., and Precup, D. (2022). Towards Continual Reinforcement Learning: A Review and Perspectives. arXiv:2012.13490 [cs].

Koza, J. R. (1994). Genetic programming as a means for programming computers by natural selection. Statistics and Computing, 4(2):87-112.

Lange, R. T. (2022). evosax: Jax-based evolution strategies. arXiv preprint arXiv:2212.04180.

Langton, C. G. (1997). Artificial Life: An Overview. MIT Press. Google-Books-ID: qErpoKjc1h4C.

Lehman, J. and Miikkulainen, R. (2015). Extinction Events Can Accelerate Evolution. PLOS ONE, 10(8):e0132886. Publisher: Public Library of Science.

Maslin, M. A., Shultz, S., and Trauth, M. H. (2015). A synthesis of the theories and concepts of early human evolution. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 370(1663):20140064.

Mirzadeh, S. I., Chaudhry, A., Yin, D., Nguyen, T., Pascanu, R., Gorur, D., and Farajtabar, M. (2022). Architecture Matters in Continual Learning. arXiv:2202.00275 [cs].

Muppidi, A., Zhang, Z., and Yang, H. (2024). Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learning.

Nisioti, E. and Moulin-Frier, C. (2022). Plasticity and evolvability under environmental variability: the joint role of fitnessbased selection and niche-limited competition. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '22, pages 113-121, New York, NY, USA. Association for Computing Machinery.

Nisioti, E., Plantec, E., Montero, M., Pedersen, J., and Risi, S. (2025). When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks? In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '25, pages 48-57, New York, NY, USA. Association for Computing Machinery.

Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. (2017). Evolution Strategies as a Scalable Alternative to Reinforcement Learning. arXiv:1703.03864 [stat].

Shelton, D. S. and Martins, E. P. (2017). Behavioral variation, adaptation, and evolution. In APA handbook of comparative psychology: Basic concepts, methods, neural substrate, and behavior, Vol. 1, APA handbooks in psychology®, pages 219-236. American Psychological Association, Washington, DC, US.

Soros, L., Joel, L., and O. Stanley, K. (2017). Open-endedness: The last grand challenge you've never heard of.

Soros, L. and Stanley, K. (2014). Identifying Necessary Conditions for Open-Ended Evolution through the Artificial Life World of Chromaria. pages 793-800. MIT Press.

Stanley, K. O. and Miikkulainen, R. (2002). Evolving Neural Networks through Augmenting Topologies. Evolutionary Computation, 10(2):99-127.

Such, F. P., Madhavan, V., Conti, E., Lehman, J., Stanley, K. O., and Clune, J. (2018). Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning. arXiv:1712.06567 [cs].

Wang, L., Zhang, X., Su, H., and Zhu, J. (2024). A Comprehensive Survey of Continual Learning: Theory, Method and Application. arXiv:2302.00487 [cs].

# Swarm Size Estimation using Diffusion Equations 

Loona Macabre ${ }^{1,2}$, Anthony J. Genot ${ }^{3}$, Leo Cazenille ${ }^{1}$, Nathanaël Aubert-Kato ${ }^{2}$, and Nicolas Bredeche ${ }^{1,4}$<br>${ }^{1}$ Sorbonne Université, CNRS, ISIR, F-75005 Paris, France<br>${ }^{2}$ Ochanomizu University, Department of Information Sciences, Tokyo, Japan<br>${ }^{3}$ LIMMS (IRL2820)/CNRS-IIS, University of Tokyo, Tokyo, Japan<br>${ }^{4}$ Sorbonne Université, CNRS, IBPS, LJP, F-75005 Paris, France


#### Abstract

This work aims to describe an algorithm to estimate the size of a swarm - i.e., the number of individuals - in a distributed manner. The idea is to replicate a chemical diffusion process (temperature, fluids...) within a swarm of robots. This article show experimental results for static and dynamic swarms of robots to demonstrate how diffusion can be used for such a task, and, more broadly, how it enables macroscopic information (such as data about the swarm and the environment) to be accessible at the microscopic scale (individual robots).


## Submission type: Late Breaking Abstracts

## Introduction

In a swarm of robots, each unit can only communicate with its immediate neighbors, making it difficult to estimate global information, such as the total number of robots or the structure of the swarm.

Our work proposes an interdisciplinary approach that combines the physics of complex systems and swarm robotics, taking inspiration from diffusion behaviors and gossip-based communication. We aim to mimic the process of chemical diffusion, by which molecules spread from an area of higher concentration to an area of lower concentration, leading eventually to a uniform distribution of particles
in the environment. If we picture a glass filled with water and place a drop of red food coloring in it, at first the red coloring is concentrated in one spot. After a while, the color becomes evenly mixed with the water: the concentration of red is the same in every area of the environment (Fig. 1). This process is described in Fick's first law:

$$
J=-D \frac{d \phi}{d x}
$$

where $J$ is the diffusion flux, which measures the amount of substance that will flow through a unit area during a unit time interval; $D$ is the diffusion coefficient; $\frac{d \phi}{d x}$ is the concentration gradient of substance $\phi$. This law ensures that, given enough time, a system will reach a steady state where the concentrations will be the same throughout (Poirier and Geiger, 2016).

## State of the art

Finding a distributed, efficient, scalable, and reliable way for robots to estimate the size of their own swarm is a fundamental task. Wang and Rubenstein (2020) propose a probabilistic solution in which each robot draws a random number between 0 and 1 , and then identifies the highest random number drawn across the entire swarm. By repeating these random draws, the average value $k$ of the maximums ob-

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-300.jpg?height=442&width=1760&top_left_y=1969&top_left_x=153)
Figure 1: Research summary. (LEFT) A swarm of Pogobots wandering around and performing swarm size estimation. (RIGHT) The four steps of the algorithm for distributed swarm size estimation using diffusion. The left snapshot corresponds to step 2.

tained in each series allows for estimating the swarm size using the formula $n^{*}=\frac{k}{1-k}$.

Brambilla et al. (2009) propose another solution loosely based on the signaling behavior of fireflies and crickets, directly inspired by the work of Melhuish et al. (1999). In their approach, each robot has a simple behavior and oscillates between a silent phase and a signaling phase.

Among the classic methods to retrieve global information locally in a multi-agent system, we can consider gossipbased algorithms such as Push-Sum and its variants. Agents estimate the size of the swarm by exchanging pairs of values $(s, w)$, where $s$ represents the accumulated sum and $w$ the associated weight, and subsequently computing the ratio $\frac{s}{w}$ (Kempe et al., 2003). Similarly to our contribution, agents diffuse information through pairwise communication, where half of their current amount is transmitted to their partner.

Our objective is to design a distributed, asynchronous, and scalable method that not only achieves fast convergence but also exhibits robustness to failures and to error propagation, which commonly affect probabilistic approaches. Taking inspiration from natural behaviors is a common and efficient approach in swarm robotics; for instance, Cazenille et al. (2024) uses diffusion processes to estimate the shape of an arena by analyzing the convergence speed.

## Algorithm

Our method consists in simulating Fick's law via pairwise communication, similarly to gossip protocols. Each robot stores a concentration value $v$ between 0 and 1 , representing a proportion of matter. Initially, the entire amount of matter is concentrated on one robot, which sets its concentration to 1 - all of the others are set to 0 . At each step, a robot communicates with its neighbors to diffuse the matter following this formula:

$$
v_{\text {new }}=v_{\text {current }}+D *\left(v_{\text {neighbor }}-v_{\text {current }}\right)
$$

A robot can then directly estimate the size of the swarm by computing $\frac{1}{v}$. In line with the properties of diffusion, this estimate converges towards the actual number of robots.

## Experimental results

We conducted experiments on real robot swarms of various topologies and sizes. The robots used are Pogobots ${ }^{1}$ : small swarm robots developed at Sorbonne Université. Pogobots have a diameter of approximately 5 cm and communicate via infrared signals. They can move using different types of locomotion, the most common of which are toothbrushhead feet with vibrating motors, and wheels powered by rotating motors (Loi et al., 2025). Here, robots are equipped with toothbrush-head feet and perform a random walk. The toothbrush heads do not allow precise locomotion and the robots may become trapped in rotational motions. However,

[^18]their design, coupled with motor vibration, facilitates trajectory changes and effective mixing in case of collisions with walls or other robots.

Each robot executes its behavior at a frequency $F$ and at each tick may transmit messages with a probability $P$ _send. This transmission probability helps reduce message collisions in high-density scenarios: the fewer the robots trying to send simultaneously, the higher the chance that each message is successfully received. Finally, the parameter $D$ is the diffusion coefficient present in Fick's law.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-301.jpg?height=440&width=830&top_left_y=744&top_left_x=1069)
Figure 2: A square with 12 static robots - Two robots are probed: the robot with initial concentration at $v=1.0$ and a random robot. 10 independent runs, average values.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-301.jpg?height=439&width=506&top_left_y=1455&top_left_x=1393)
Figure 3: 12 robots randomly walking in a $40 \times 47 \mathrm{~cm}$ arena - One external robot acts as a probe, collecting data sent by nearby robots. 1 typical run.

Experiments demonstrate that the diffusion method provides an accurate estimate of the swarm size, both in static (Fig. 2) and dynamic (Fig. 3) swarms. The initial spot of concentration diffuses in the swarm and converges to the correct value and estimate. They also highlight how movement influences convergence speed: with 12 robots randomly moving within an arena, convergence is much faster than with static robots. While different topologies may affect the rate of convergence, results show that movement accelerates diffusion, similarly to physical diffusion.

## Acknowledgements

This work was supported by the SSR project funded by the Agence Nationale pour la Recherche under Grant No ANR-24-CE33-7791 and JSPS Grant-in-Aid for Scientific Research (A) No JP25H00608.

## References

Brambilla, M., Pinciroli, C., Birattari, M., and Dorigo, M. (2009). A reliable distributed algorithm for group size estimation with minimal communication requirements. In ICAR, pages 1-6.

Cazenille, L., Lobato-Dauzier, N., Loi, A., Macabre, L., Ito, M., Marchal, O., Kim, S.-H., Aubert-Kato, N., Bredeche, N., and Genot, A. J. (2024). Hearing the shape of an arena with spectral swarm robotics. arXiv preprint arXiv:2403.17147.

Kempe, D., Dobra, A., and Gehrke, J. (2003). Gossip-based computation of aggregate information. In 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings., pages 482-491.

Loi, A., Macabre, L., Fersula, J., Amini, K., Cazenille, L., Caura, F., Guerre, A., Gourichon, S., Dauchot, O., and Bredeche, N. (2025). Pobogot - an open-hardware open-source low cost robot for swarm robotics.

Melhuish, C., Holland, O., and Hoddell, S. (1999). Convoying: using chorusing to form travelling groups of minimal agents. Robotics and Autonomous Systems, 28(2):207-216. Intelligent Autonomous Systems (IAS-5).

Poirier, D. and Geiger, G. (2016). Fick's Law and Diffusivity of Materials, pages 419-461.

Wang, H. and Rubenstein, M. (2020). A fast, accurate, and scalable probabilistic sample-based approach for counting swarm size. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 7180-7185.

# Learning Global Rules from Local Patches: Scaling Neural Cellular Automata 

Aleksandr Groznykh ${ }^{1,2}$<br>¹California Institute for Machine Consciousness, USA<br>${ }^{2}$ Brandenburg University of Technology, Germany<br>groznykh.aleksandr@gmail.com

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-304.jpg?height=396&width=839&top_left_y=831&top_left_x=155)
Figure 1: (Left) Local image patches used in NCA training. (Right) Generalization of the learned rule to full-scale image. The top row is RGB state, other rows are internal cellular states of the NCA.


#### Abstract

Neural Cellular Automata (NCA) represent a compelling model of bio-inspired distributed computation where a uniform grid of agents achieves global objectives through local communication rules. These systems apply a single differentiable update rule iteratively across the grid, with parameters learned through gradient descent via backpropagation through time (BPTT). However, BPTT creates a fundamental scalability bottleneck: memory requirements grow quadratically with grid size, quickly saturating available GPU memory and limiting practical applications. This study investigates whether NCAs can scale beyond their training dimensions by learning from local patches rather than entire target configurations. We train NCAs on small patches extracted from larger target images and evaluate whether the learned rules generalize to full-scale problems. Our results demonstrate that patch-trained NCAs successfully reconstruct complete images, suggesting that local update rules can capture global coordination patterns. This finding has direct implications for scaling NCA training to larger, more complex problems while maintaining computational feasibility, potentially enabling exploration of emergent behaviors previously constrained by memory limitations.

Submission type: Late Breaking Abstract Code available at: Jupyter notebook


## Introduction

Artificial life seeks to understand how simple, local interactions give rise to complex global behaviors-a principle exemplified throughout biological systems.

Neural Cellular Automata (Mordvintsev et al., 2020) embody this principle by implementing uniform computational rules across spatial grids, where each cell's behavior depends only on its local neighborhood. However, current NCA training methodologies face a scalability challenge: memory requirements scale quadratically with grid dimensions due to the need to store intermediate states for BPTT (Pajouheshgar et al., 2025). This constraint limits practical applications to relatively small grids, preventing exploration of more complex emergent behaviors that might arise in larger systems. Naively reducing the number of timesteps per training epoch hinders the quality of learned representations and reduces long-term stability. While NCAs should theoretically scale to arbitrary sizes due to their uniform rule structure, existing training approaches require full-grid backpropagation, creating a memory bottleneck for largescale applications.

Living organisms demonstrate remarkable efficiency in processing complex environments through localized attention, focusing computational resources on relevant patches of sensory input rather than processing entire scenes simultaneously. Our approach takes inspiration from this mechanism and leverages the inherent uniformity of NCAs: if the update rule is truly general, it should be learnable from representative patches of the target configuration rather than requiring full-scale training data.

## Methods

Our method extends Texture Neural Cellular Automata (Niklasson et al., 2021), applying it to image generation using patch-based loss. Instead of applying backpropagation across entire target images, we train the NCA using MSE loss computed on local patches extracted from the target image. This modification enables training on larger images while maintaining fixed memory requirements. The training procedure operates as follows:

Patch Sampling Strategy: For each training epoch, we sample square patches from the target image. To ensure diverse gradient signals within each batch, we enforce spatial diversity by sampling patches from different image regions. This prevents the NCA from learning redundant representations of similar local patterns within a single training step. Between epochs, patch locations are gradually shifted. This ensures that the network observes different local patches extracted from larger target images throughout training, preventing overfitting to specific spatial locations while maintaining coverage of the entire target domain.

Boundary Handling: During NCA evolution within patches, boundary cells (those with fewer than 8 neighbors due to patch edges) are excluded from updates and the loss calculations. This design choice can be interpreted as implementing a self-organized rule where cells only update when surrounded by a complete neighborhood. This boundary treatment reduces artifacts from patch edges while maintaining the local nature of cellular interactions.

Rule Update: After each epoch, the $M S E$ loss is computed by comparing the final state of each patch against the corresponding region from the target image. Gradients are backpropagated through the NCA unfolding sequence.

## Results \& Discussion

Generalization Success: Our experiments demonstrate that locally-trained NCAs successfully generalize to reconstruct complete target images. Despite being trained exclusively on $P \times P$ patches ( $P=64$ in our experiments), the learned rules coordinate cell behaviors across global scale $S$ ( $S=$ 128 in our experiments), producing coherent global patterns that match the target configuration.

Scaling Implications: NCA training memory requirements scale as $O\left(N * P^{2} T\right)$ rather than $O\left(N * S^{2} T\right)$, where $N$ is the batch size and $T$ is the number of timesteps. For a $128 \times 128$ target image trained with $64 \times 64$ patches, the spatial reduction alone decreases memory by $4 \times$. Moreover, in our experiments, smaller NCAs were able to be trained on proportionally fewer timesteps ( $2 \times$ reduction), yielding a total 8 -fold reduction in memory requirements, without losing representation details and long-term stability.

Emergent Coordination: The success of patch-based training suggests that NCA rules can encode coordination mechanisms that extend beyond their training context. Local update rules learned from patches appear to contain sufficient information to orchestrate global pattern formation, supporting the hypothesis that uniform cellular automata can achieve scalable computation through purely local interactions.

This patch-to-global generalization capability opens new possibilities for studying emergence and self-organization in Neural Cellular Automata previously constrained by computational limitations.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-305.jpg?height=648&width=808&top_left_y=301&top_left_x=1091)
Figure 2: NCA Patch-Based Training Algorithm

## Acknowledgments

This work was supported by the California Institute for Machine Consciousness.

We'd like to thank Linus Eickhoff for thoughtful proofreading and giving extensive feedback.

## References

Mordvintsev, A., Randazzo, E., Niklasson, E., and Levin, M. (2020). Growing neural cellular automata. Distill. https://distill.pub/2020/growing-ca.

Niklasson, E., Mordvintsev, A., Randazzo, E., and Levin, M. (2021). Self-organising textures. Distill. https://distill.pub/selforg/2021/textures.

Pajouheshgar, E., Xu, Y., Abbasi, A., Mordvintsev, A., Jakob, W., and Süsstrunk, S. (2025). Neural cellular automata: From cells to pixels. arXiv preprint.

# Gridarians: An LLM-Driven Artificial Life Platform for Exploring Brain-Body Coevolution 

Can Gurkan ${ }^{1}$ and Uri Wilensky ${ }^{1}$<br>${ }^{1}$ Northwestern University, Evanston, IL<br>gurkan@u.northwestern.edu


#### Abstract

Artificial Life (ALife) offers a powerful lens for studying the coevolution of intelligence and morphology. Traditional ALife platforms often rely on hand-crafted rules or evolutionary algorithms such as genetic programming, where achieving effective brain-body co-optimization remains a challenge. We introduce Gridarians, a grid-based ALife world implemented in NetLogo, in which Large Language Models (LLMs) serve as generative engines for both the "brains" (decision rules expressed in code) and "bodies" (morphologies) of virtual creatures. In this setting, LLMs mutate and expand the representational space of agent controllers and body plans, enabling coevolutionary dynamics less constrained by pre-defined encodings. We investigate how LLM-driven variation influences the trajectory of brain-body coevolution by analyzing both behavioral outcomes and representational properties of evolved agents. Our results demonstrate that LLMs can facilitate the discovery of more diverse and robust brain-body couplings compared to Cartesian Genetic Programming, underscoring their potential as open-ended search operators in ALife research. By positioning LLMs as engines of generative variation, Gridarians opens new avenues for exploring the relationship between embodiment, intelligence, and artificial evolution.


## Submission type: Late Breaking Abstract

Data/Code available at: https://github.com/ can-gurkan/gridarians

## Introduction

The study of brain-body coevolution in Artificial Life shows that intelligence is inseparable from the morphologies through which agents perceive and act (Brooks, 1999; Pfeifer and Bongard, 2006). Yet most computational approaches struggle with effective exploration and the openended coevolution of brains and bodies (Kudithipudi et al., 2022; Bongard, 2011; Bongard et al., 2018). Although numerous studies have demonstrated the potential of embodied evolution and enabled progress, they remain constrained by fixed encodings, narrowly defined mutation operators, and the costly evaluations required to assess morphological fitness, which often obscure promising innovations (Mertan and Cheney, 2025; Lai et al., 2021; Liu et al., 2025). Large

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-306.jpg?height=512&width=515&top_left_y=829&top_left_x=1238)
Figure 1: The Gridarians platform, in which agents are composed of cell-based morphologies and controller programs, both evolved through LLMs.

language models (LLMs), by contrast, offer powerful generative capabilities for producing structured code and abstractions that transcend limited representations and are beginning to replace hand-designed search heuristics (Brown et al., 2020; Hu et al., 2025; Lu et al., 2024; Fernando et al., 2024; Hagos et al., 2024). Despite their growing use in code generation and program synthesis (Liu et al., 2024; Romera-Paredes et al., 2024), LLMs remain largely unexplored in ALife (Nisioti et al., 2024; Kumar et al., 2025), even though they have the potential to serve as powerful engines of variation (Lehman et al., 2024; Hemberg et al., 2024; Gurkan et al., 2025). Their semantic understanding makes it possible to generate agent controllers tailored to evolving morphologies by conditioning on body structure as context, avoiding the need for costly retraining whenever morphologies change (Song et al., 2025; Qiu et al., 2025). This opens new opportunities to revisit brain-body coevolution, enabling more fluid co-adaptation of controllers and bodies. In this work we introduce Gridarians, a grid-based ALife platform that leverages LLMs to coevolve controllers and morphologies, providing a novel testbed for investigating how LLM-driven generative processes reshape the dynamics of embodied evolution.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-307.jpg?height=531&width=549&top_left_y=267&top_left_x=294)
Figure 2: Graphical overview of the LLM-driven mutation operator for agent body and code.

## Methods

We implemented a grid-based Artificial Life platform in NetLogo (Wilensky, 1999) in which agents are composed of discrete cells. Each cell is defined by x and y coordinates, a cell type, and a facing direction. Four cell types serve as the fundamental building blocks of morphology: sensor, movement, rotation, and interaction. Sensor cells detect the nearest non-empty grid square in the direction they are facing. Movement cells translate the agent forward, rotation cells pivot the body $90^{\circ}$ around a fixed seed cell at ( 0,0 ), and interaction cells consume adjacent food items automatically. Bodies are encoded as lists of four-tuples specifying relative coordinates, cell type, and direction of each cell.

Behavior is controlled by an evolved NetLogo program that maps sensory inputs to actions permitted by the morphology. At each time step, sensor cells produce an observation vector, which the controller program uses to issue actions corresponding to available movement and rotation cells. The environment also contains walls and renewable food sources as shown in Figure 1. Agents try to maximize food collection within a fixed time horizon. When a food item is consumed, another spawns at a random location.

LLM-Driven Mutation At each generation, the bestperforming agents reproduce by generating mutated offspring that replace the least fit individuals. To employ LLMs as mutation operators, we use a structured multi-step process. First, the agent body, encoded as a list of integers, is provided to the LLM with prompts designed to elicit meaningful structural variations. The resulting mutated body, together with the parent's controller program, is then supplied to the LLM, which produces a modified program conditioned on the new morphology. Figure 2 illustrates this mutation pipeline for bodies and controllers. Mutated agents are subsequently evaluated in the simulation environment based on their food-collection performance. Before execution, all generated programs undergo verification to ensure syntactic correctness and domain relevance.

## Experiments \& Results

In preliminary experiments, we compared the mean fitness of agents evolved through the LLM-driven mutation pipeline with those evolved using Cartesian Genetic Programming (CGP) (Miller, 2011) to represent bodies and controllers. As shown in Figure 3, results indicate that compared to CGP, the LLM-driven approach produced agents with significantly higher fitness (amount of food collected), underscoring the suitability of LLM-powered variation operators with text-based representations for coevolving brains and bodies.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-307.jpg?height=497&width=663&top_left_y=758&top_left_x=1160)
Figure 3: Mean fitness of the agents over generations for LLM-driven and CGP mutation.

## Discussion

Our experimental results point toward several broader implications. First, positioning LLMs as mutation operators expands the representational space beyond fixed encodings, enabling the discovery of novel couplings between morphology and control that might otherwise remain inaccessible. Second, the ability of LLMs to condition program generation on specific morphological configurations suggests a path toward more fluid co-adaptation, where changes to the body do not necessitate costly retraining of the controller.

At the same time, LLM-driven mutation remains highly sensitive to prompt design, with open challenges in ensuring reliability, semantic alignment, and behavioral plausibility. Moreover, our evaluation was limited to a simple foodcollection task; more complex environments will be needed to assess the generality of these benefits.

## Conclusion

This work presents one of the first attempts to integrate LLM-driven code evolution into ALife, demonstrating their potential to coevolve agent morphologies and controllers within a unified framework. Our findings point to new opportunities for studying brain-body coevolution and highlight the promise of LLMs as generative engines in ALife, opening pathways to novel explorations of embodiment and intelligence.

## References

Bongard, J. (2011). Morphological change in machines accelerates the evolution of robust behavior. Proceedings of the National Academy of Sciences, 108(4):1234-1239.

Bongard, J. C., Cheney, N., Mahoor, Z., and Powers, J. P. (2018). The role of embodiment in open-ended evolution.

Brooks, R. A. (1999). Cambrian Intelligence: The Early History of the New AI. The MIT Press.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. (2020). Language models are few-shot learners. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.

Fernando, C., Banarse, D. S., Michalewski, H., Osindero, S., and Rocktäschel, T. (2024). Promptbreeder: Self-referential selfimprovement via prompt evolution. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F., editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 1348113544. PMLR.

Gurkan, C., Jwalapuram, N. K., Wang, K., Danda, R., Rasmussen, L., Chen, J., and Wilensky, U. (2025). Lear: Llm-driven evolution of agent-based rules. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO '25 Companion, page 2309-2326, New York, NY, USA. Association for Computing Machinery.

Hagos, D. H., Battle, R., and Rawat, D. B. (2024). Recent advances in generative ai and large language models: Current status, challenges, and perspectives. IEEE Transactions on Artificial Intelligence, 5(12):5873-5893.

Hemberg, E., Moskal, S., and O'Reilly, U.-M. (2024). Evolving code with a large language model. Genetic Programming and Evolvable Machines, 25(2):21.

Hu, S., Lu, C., and Clune, J. (2025). Automated design of agentic systems. In The Thirteenth International Conference on Learning Representations.

Kudithipudi, D., Aguilar-Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A., Chakravarthi Raja, S., Cheney, N., Clune, J., Daram, A., Fusi, S., Helfer, P., Kay, L., Ketz, N., Kira, Z., Kolouri, S., Krichmar, J., Kriegman, S., and Siegelmann, H. (2022). Biological underpinnings for lifelong learning machines. Nature Machine Intelligence, 4:196-210.

Kumar, A., Lu, C., Kirsch, L., Tang, Y., Stanley, K. O., Isola, P., and Ha, D. (2025). Automating the search for artificial life with foundation models.

Lai, G., Leymarie, F., Latham, W., Arita, T., and Suzuki, R. (2021). Virtual creature morphology - a review. Computer Graphics Forum, 40(2):659-681.

Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., and Stanley, K. O. (2024). Evolution Through Large Models, pages 331366. Springer Nature Singapore, Singapore.

Liu, F., Tong, X., Yuan, M., Lin, X., Luo, F., Wang, Z., Lu, Z., and Zhang, Q. (2024). Evolution of heuristics: towards efficient automatic algorithm design using large language model. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org.

Liu, H., Guo, D., and Cangelosi, A. (2025). Embodied intelligence: A synergy of morphology, action, perception and learning. ACM Comput. Surv., 57(7).

Lu, C., Holt, S., Fanconi, C., Chan, A. J., Foerster, J., van der Schaar, M., and Lange, R. T. (2024). Discovering preference optimization algorithms with and for large language models. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., and Zhang, C., editors, Advances in Neural Information Processing Systems, volume 37, pages 8652886573. Curran Associates, Inc.

Mertan, A. and Cheney, N. (2025). Evolutionary brain-body cooptimization consistently fails to select for morphological potential.

Miller, J. F. (2011). Cartesian Genetic Programming, pages 17-34. Springer Berlin Heidelberg, Berlin, Heidelberg.

Nisioti, E., Glanois, C., Najarro, E., Dai, A., Meyerson, E., Pedersen, J. W., Teodorescu, L., Hayes, C. F., Sudhakaran, S., and Risi, S. (2024). From text to life: On the reciprocal relationship between artificial life and large language models. volume ALIFE 2024: Proceedings of the 2024 Artificial Life Conference of Artificial Life Conference Proceedings, page 39.

Pfeifer, R. and Bongard, J. (2006). How the Body Shapes the Way We Think: A New View of Intelligence. The MIT Press.

Qiu, K., Pałucki, W., Ciebiera, K., Fijałkowski, P., Cygan, M., and Łukasz Kuciński (2025). Robomorph: Evolving robot morphology using large language models.

Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, P., Fawzi, O., Kohli, P., and Fawzi, A. (2024). Mathematical discoveries from program search with large language models. Nature, 625(7995):468-475.

Song, J., Yang, Y., Xiao, H., Peng, W., Yao, W., and Wang, F. (2025). LASer: Towards diversified and generalizable robot design with large language models. In The Thirteenth International Conference on Learning Representations.

Wilensky, U. (1999). NetLogo. http://ccl.northwestern.edu/netlogo/, Northwestern University, Evanston, IL.

# Towards Internalist and Socio-Affective Models of Cultural Evolution 

Erwan Plantec ${ }^{1}$, Jayd Matyas ${ }^{2}$, Eva Mardus ${ }^{3}$, Yoko Urano ${ }^{4}$ and Daan Laméris ${ }^{5}$<br>${ }^{1}$ Robotics Evolution and Artificial Life lab, IT University, Copenhagen, Denmark ${ }^{2}$ DeepMind, London, UK<br>${ }^{3}$ Natural History Museum, Berlin, Germany, ${ }^{4}$ University of California, Berkeley, US ${ }^{5}$ Indiana University, Bloomington, US<br>erpl@itu.dk, jmatyas@google.com


#### Abstract

Computational models of cultural evolution have largely emphasised external dynamics, depicting agents as passive elements. We present an agent model that integrates socioemotional dynamics and internal constraints and motivations into cultural learning. Agents encode behavioural distributions with a variational autoencoder; behaviours that are observed and learned by other agents through a reconstructive process. We investigate the emergent dynamics of our model with and without environmental feedback and demonstrate its ability to reproduce well-studied opinion dynamics under bounded confidence.


## Introduction

Computational models of cultural evolution (CE) have historically focused on external factors and transmission dynamics, often casting individual agents as passive transmission nodes and cultural artifacts as discrete gene-like structures, called memes, subject to random variation (Henrich and Boyd, 2002; Boyd and Richerson, 1988). This simplified view omits internal constraints and relational dynamics despite clear evidence of their importance in shaping behaviour across human and non-human animals (Russell, 2003; Schacter, 2012; De Waal, 2008). Such simplifications risk overlooking how cultural change arises from subjective and socially embedded processes (Perry et al., 2021). We seek to extend this tradition by centring socio-affective dynamics: treating CE as an emergent property of agents' emotional states social dynamics and internal constraints and motivations.

## Model

Culture can be defined as a set of practices, skills, or more generally, behaviours that are shared among a population and transmitted via social learning. From this definition, we model the cultural repertoire of agents as probability distribution in a behavioural space $\mathcal{B}$ which can be be interpreted as the probability that an agent engage in any behaviour, its steady-state density. We note $\mathbf{b}_{i}=\mathbb{P}(b \in \mathcal{B})$ the behavioural distribution of agent $i$ and $\mathbf{b}$ the joint behavioural distribution among the entire population. Agents distributions
are modelled with Variational Autoencoders (VAE; Kingma and Welling (2019)) whose decoder $d_{i}$ (for agent $i$ ) maps a latent space prior distribution $\mathbf{p}=\mathcal{N}^{L}(0,1)$ to the agent behavioural distribution $\mathbf{b}_{i}=d_{i}(\mathbf{p})$ where $L$ is the latent space dimensionality. Importantly, VAEs parameters are not shared by agents.
From this, social learning can be modelled by using externally observed behaviours, e.g. behaviours displayed by other agents, as training data for agents' VAE parameters. Let's note $A=\left\{a_{i j}\right\}$ the adjacency matrix encoding agents social connections to each other ( $a_{i j}=0$ meaning no connection between agents $i$ and $j$ ). At each simulation step, agents observe a set of behaviours sampled from the combined distributions of their neighbours $\left(\mathbf{o}_{i}(t)=P_{i j} \times \mathbf{b}_{j}(t)=\right.$, with $P_{i j}$ the probability that agent $i$ samples from agent $j$ ). Agents can then update their encoder and decoder parameters using $\mathbf{o}_{i}(t)$ as training data.
One appeal of our approach is that it provides natural ways to model exploratory behaviours, e.g. playfulness, allowing to account for the active role of agents in producing culture. This can be done by modulating how agents sample behaviours from their distribution which are then observed by other agents. For instance, positive affective states has been show to be correlated with more playful/exploratory behaviours Perry et al. (2021). Playfulness can be modelled by modulating the shape of the latent prior distribution $\mathbf{p}$ for instance, by scaling its variance as a function of the agent affective state thus modulating the variance of the behavioural distribution (see fig.1-a)
External, or environmental, feedback can also be naturally modelled in this setting. Let's note $\mathcal{E}: \mathcal{B} \rightarrow \mathbb{R}$ the environment, or fitness, function which associates to each behaviour a return or reward. The total return (or fitness) of an agent at time $t$ is given by the expected return given the agent behavioural distribution: $r_{i}=\mathbb{E}_{b \sim \mathbf{b}_{i}}[\mathcal{E}(b)]$. In practice, this is evaluated by sampling a finite set of behaviours from the agent distribution.
Social learning biases are central to CE dynamics and can modulate social learning based on context (who is teaching) or content (what is learnt) (Henrich and McElreath, 2003).

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-311.jpg?height=686&width=1514&top_left_y=274&top_left_x=248)
Figure 1: $\mathbf{a}$ Agents' VAE encode distributions in behaviour space (here $\mathbb{R}^{2}$ ). The level of playfulness of agents can be modulated (from low (purple) to high (yellow)) resulting in more or less exploration of their behavioural repertoire. b Clustering of behaviours emerge under bounded confidence bias; black dots correspond to the means of each agent behavioural distribution while red correspond to the end of simulation. c When introducing environmental feedback and prestige bias, agents' tend to internalize the environments structure (bottom-right: individual agents' distributions is elongated along the axis of maximum improvement) leading to an acceleration of improvement. top-left: variance of population-level distribution. top-right: average and maximum returns. bottom-left(right): agents distributions before (after) simulation. d Iterated learning schemes can be used to uncover the agents' internal biases.

Our model allows to integrate such biases by (i) modulating how agents sample from other agents allowing to model context based biases or (ii) weighting differentially the training samples in the loss function either based on context or content.

## Results

We observe cultural collapse, i.e the population level distribution $\mathbf{b}$ approaching 0 variance, under conformist dynamics (unbiased agents uniformly sampling and learning from their neighbours) and when the social network contains cycles. We only observed non-collapsing cyclic dynamics when connecting agents on an unidirectional ring.
When adding environmental feedback $\left(\mathcal{E}\left(b \in \mathbb{R}^{2}\right)=b_{0}-\right. \left.b_{1}\right)$ together with prestige bias ( $P_{i j}=\frac{e^{r_{j}}}{\sum e^{r^{k}}}$ ), the average agent return does increase over time. More importantly, we observe an acceleration (more than linear improvement, where the environment function is linear; see fig.1-c) indicating that agents have internalized the environment structure, a phenomenon similar to the evolution of evolvability Pigliucci (2008). Interestingly, we also observed a phase of decreasing fitness corresponding to agents drifting along the $y$-axis. Such phenomenon indicates the presence of agents internal biases which under certain condition can even lead to maladaptive dynamics.
Such internal biases can be uncovered using the iterated
learning schema (Kirby et al., 2014) where an initial agent is taught ground truth distribution and has to teach a new agent with no prior knowledge, then this new agent must teach anew one and so on. Starting from a ground truth composed of 4 gaussians forming a cross-like pattern, we observed that distributions tend to get larger and, more importantly, drift towards a $y=x$ relation which could explain the aforementioned maladaptive drift (see fig.1-d).
Bounded confidence is a well studied social learning bias where agents learn from each other only if their opinions are close enough (Rainer and Krause, 2002). We implement such a mechanism by preventing agents to learn from each other if the distance between the means of their respective behavioural distribution exceeds some threshold. Under such a bias, agents tend to form clusters (see fig.1-b) under the right threshold value (here 0.05 ).

## Discussion and Future Work

These preliminary results show the potential of our model as a base for constructing a more complete, internalist account of CE. Our next step is to add affective modulation, emotional contagion, and relationship bonds, which we expect to drive social self-organization and play an important role in shaping CE and generating diversity. Future work should also focus on reproducing empirical data, examine internalisation under shifting environments, and probe how affective dynamics sustain resilience and diversity.

## Acknowledgements

Funded by the European Union (ERC, GROW-AI, 101045094).

## References

Boyd, R. and Richerson, P. J. (1988). Culture and the Evolutionary Process. University of Chicago Press, Chicago, IL.

De Waal, F. B. (2008). Putting the altruism back into altruism: the evolution of empathy. Annu. Rev. Psychol., 59(1):279-300.

Henrich, J. and Boyd, R. (2002). On modeling cognition and culture: Why cultural evolution does not require replication of representations. Journal of Cognition and Culture, 2(2):87112. Place: United Kingdom Publisher: Brill Academic Publishers.

Henrich, J. and McElreath, R. (2003). The evolution of cultural evolution. Evolutionary Anthropology: Issues, News, and Reviews, 12(3):123-135. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/evan.10110.

Kingma, D. P. and Welling, M. (2019). An Introduction to Variational Autoencoders. Foundations and Trends® in Machine Learning, 12(4):307-392. Publisher: Now Publishers, Inc.

Kirby, S., Griffiths, T., and Smith, K. (2014). Iterated learning and the evolution of language. Current Opinion in Neurobiology, 28:108-114.

Perry, S., Carter, A., Smolla, M., Akçay, E., Nöbel, S., Foster, J. G., and Healy, S. D. (2021). Not by transmission alone: the role of invention in cultural evolution. Philosophical Transactions of the Royal Society B, 376(1828):20200049.

Pigliucci, M. (2008). Is evolvability evolvable? Nature Reviews Genetics, 9(1):75-82. Publisher: Nature Publishing Group.

Rainer, H. and Krause, U. (2002). Opinion Dynamics and Bounded Confidence: Models, Analysis and Simulation. Journal of Artificial Societies and Social Simulation, 5(3).

Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological review, 110(1):145.

Schacter, D. L. (2012). Adaptive constructive processes and the future of memory. American Psychologist, 67(8):603.

# Uiua: A Modern Array Language for Artificial Life Research 

Noah Syrkis<br>IT University of Copenhagen<br>nobr@itu.dk


#### Abstract

Uiua ${ }^{1}$, an APL-like stack-based array programming language, offers a powerful alternative for Artificial Life (ALife) research. Its concise syntax and efficient array operations enable rapid prototyping and complex simulations of bio-like systems. This abstract showcases Uiua's capabilities through a basic particle swarm simulation example and discusses its potential for ALife research.


## Introduction

Artificial Life (ALife) research often involves complex simulations of bio-like systems, which require efficient manipulation of large datasets and intricate computational models. Traditional programming languages, while versatile, may not always provide the optimal balance of expressiveness and performance for such tasks. Uiua, an APL-like stack-based array language written in Rust, offers a compelling alternative for ALife researchers.

All operations operate on a global stack. For examples ++123 pushes the values 3 , 2 , and 1 onto the stack while the left most + adds 3 to the sum of 1 and 2 . Uiua's syntax and semantics are designed to be intuitive and concise, allowing researchers to express complex algorithms with minimal code. Plugging these expressions into high level functional combinators, a subset of which is described in Listing 1, can be used to create powerful and efficient ALife simulations.

| ^ fold | $\underline{\circ}$ under | 三 rows |
| :--- | :--- | :--- |
| Apply a function to aggregate arrays | Operate on a transformed array, then untransform it | Apply a function to each row of an array or arrays |
| $\Lambda+\left[\begin{array}{lll}1 & 2 & 3\end{array}\right] 10 \rightarrow 16$ | $\underline{0+(\times 2)} \quad 15 \rightarrow 11$ | $\equiv \Lambda+\left[1 \_2\right.$ 4_5] $0 \rightarrow 3 \_9$ |

Listing 1: Combinators (top) with descriptions (mid) and examples (bottom)

Note how each function in Listing 1 takes in another function as an argument. In the context of ALife, this other function could be a step function. Uiua's ability to perform vectorized operations on entire arrays at once can significantly reduce computational overhead, while allowing seamless abstraction over low-level details. For example, the Step for Conway's game of life can be written as Step $\leftarrow 1 \cap^{\prime}=3 \circledast-,: /+\underline{1} \_\infty \circlearrowright / \subset-1 \hat{\imath}$ 3_3. (by the user Garmelon on the very welcoming Uiua discord server).

[^19]
## Method

The following example demonstrates a simple particle swarm simulation in Uiua. Frist we define Init that, awaiting a seed, creates a 10 by 2 array (representing position) of numbers uniformly distributed between 0 and the map size $W$. Next we define Step to take in an array and add random noise to each element, clipping all values to be within the bounds of the map. Finally Draw turns an array of positions into a location matrix (which can be used with Uiua's inbuilt \&gifs or \&ims to generate gifs or images).

```
W \leftarrow 100 # size constant
Init \leftarrow |1 + \div 2 W gen N_2 # inits
Step \leftarrow |2 i 0 i - 1 W + -0.5 ogen o-\Delta : # step
Draw \leftarrow |1 \textrm{O(g|-1) | : + 1 \times 0 \mp@subsup{}{}{\circ}\Delta W W # shows}
```

Listing 2: Init, Step and Draw functions for a basic particle swarm simulation

The vertical bar and number (|1) after the function assignment operators describes the number of arguments each function takes. Increasing the complexity of our simulations then amounts to merely elaborating the Init and Step functions.

## Results

Combining the code in Listing 4 we can run a full simulation (or arbitrarily many parallel simulations).

```
#draw ^(.Step) } 1000 Init 0 # Run and then draw sim
Save \leftarrow &fwa : img "png" # save function
\equivSave : {"1" "2" "3" "4" "5"} c o = 0 Δ 200 \uparrow 1000 # actually save
```

Listing 4: Running and saving (see Figure 1) a 1000 step simulation (with seed 0 )

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-315.jpg?height=181&width=1479&top_left_y=1716&top_left_x=324)
Figure 1: Every 200th step throughout the simulation

For more complex example, consider evolutionary neural network implementation [1] made by the creator of Uiua, which demonstrates Uiua's potential for implementing advanced ALife techniques.

## Conclusion

The particle swarm simulation demonstrates Uiua's capabilities for ALife research. By leveraging Uiua's array operations and functional programming paradigms, researchers can create efficient and expressive simulations of bio-like systems. The concise syntax and stack-based approach enable rapid prototyping and iteration, facilitating hypothesis testing and model refinement. By adopting Uiua, ALife researchers can streamline their workflows, reduce boilerplate code, and focus more on the scientific challenges at hand.

## References

[1] Kai Schmidt, "Evonet: Basic Evolutionary Neural Network in Uiua." 2025.

# Investigating the effect of network topology in Continual Reinforcement Learning 

Kathrin Korte ${ }^{1}$, Eleni Nisioti ${ }^{1}$ and Sebastian Risi ${ }^{1}$<br>${ }^{1}$ IT University of Copenhagen<br>kort, enis, sebr@itu.dk


#### Abstract

Biological neural systems exhibit sparse, structured topologies, such as modular organisation and small-world connectivity, which are believed to facilitate efficient learning and adaptability. In contrast, deep reinforcement learning (RL) networks are typically fully connected and layered, lacking such structural biases. In order to address the continual reinforcement learning problem, we examine the impact of biologically inspired connectivity in RL by comparing networks exhibiting small-world and modular properties with fully connected MLP baselines in classic Gym control tasks. We observe that modularly connected networks achieve the highest cumulative rewards across the tested environments, demonstrating an advantageous balance between retention and plasticity.


Submission type: Late Breaking Abstract

Data/Code available at: https://github.com/ kat-ko/topology-playground

## Introduction

Understanding how humans are able to perform a diverse set of complex tasks is a central question for the Artificial Intelligence community (Chaganty et al., 2012). Instead of viewing learning as finding a solution, we can instead think of it as endless adaptation (Abel et al., 2023). Biological adaptive organisms exhibit it to various extents, with some species transferring skills across increasingly complex tasks and others exhibiting a remarkably ability to switch among widely diverse behaviors within minutes (Shelton and Martins, 2017). Artificial agents, on the other hand, are classically trained in stationary environments and often fail catastrophically when encountering novel situations (Farebrother et al., 2018). Failure can come in the form of either catastrophic forgetting, where the environmental change harms previously acquired skills or plasticity loss, where the change does not induce the necessary behavioral change. These challenges have been studied in both supervised and reinforcement learning tasks, with approaches often focusing on improving the optimisation objective, the
training procedure or the architecture of the artificial neural networks (Khetarpal et al., 2022; Wang et al., 2024; Mirzadeh et al., 2022). Biological networks have evolved to be highly functional tackling both of these challenges within uncertain environments, while remaining extremely adaptable. An investigation into why biological networks are resistant to Catastrophic Forgetting finds a dependence on network topology, with healthy neural networks exhibiting small world, scale-free, and modular characteristics. (Sugimoto et al., 2025; Hintze and Adami, 2008; Early, 2019) One of the main contributors to the robustness and evolvability of biological networks is believed to be their modularity of function Hintze and Adami (2008). On the other hand small-world topologies are ubiquitous in natural and biological networks, such as the brain, and have been shown to exhibit increased computational efficacy, efficiency, and robustness (McAllister et al., 2022). While existing humancrafted network structures may enhance performance, they often remain constrained by inherent paradigms. (Pan et al., 2024) To empower models to autonomously discover optimal network architectures, the field of Neural Architecture Search (NAS) has emerged. Pan et al. (2024) In this work, instead of automatically exploring architectures and optimizing the search to find the best architecture, we directly compare different topologies, to isolate the effect of changing only feature. Driven by the inability of classic FFN to alleviate catastrophic forgetting in continual RL settings, we investigate the role of biologically inspired connectivity in RL by comparing small-world, modular, and hybrid (combination of characteristics of modular and small-world) topologies against an MLP baseline on classic gym control tasks.

## Methodology

Topologies We implemented four distinct neural network topologies to investigate the relationship between network structure and learning performance in reinforcement learning tasks (see our online repository for the hyperparameters and scripts used to reproduce the study). Modular networks are constructed by partitioning nodes into distinct modules
with controlled connectivity patterns. Each module contains $\frac{n}{M}$ nodes, where $n$ is the total network size and $M$ is the number of modules. Intra-module connections occur with probability $p_{\text {intra }}$, while inter-module connections occur with probability $p_{\text {inter }}$. Small-world networks are generated using the Watts-Strogatz model with parameters $k$ (nearest neighbors) and $p$ (rewiring probability). Hybrid networks combine small-world and modular properties by implementing small-world connectivity within modules while maintaining sparse inter-module connections. (Albert and Barabási, 2002) Hybrid Networks: Hybrid networks combine small-world and modular properties by implementing small-world connectivity within modules while maintaining sparse inter-module connections. Multilayer Perceptrons (MLPs): Dense, layer-wise connectivity across hidden layers, with each neuron connecting to all units in the next layer, serve as the standard baseline in our comparison.

Tasks We evaluated network performance across three classic control tasks from the Gymnasium environment: CartPole-v1 - A pole-balancing task requiring continuous control with binary actions. Acrobot-v1 - A twolink pendulum swing-up task with continuous state space. LunarLander-v2 - A lander navigation task requiring thrustbased control to reach a target pad with minimal fuel and soft landing. These tasks represent different control challenges, discrete vs. continuous control, varying state space complexity, and different temporal dynamics.

Experimental Setup The experimental setup follows the continual RL framework of Muppidi et al. (2024). We train agents with PPO, and introduce controlled perturbations to emulate distributional shifts. Specifically, every 200 intervals ( $\sim 160000$ PPO env. timesteps per noise level), we add Gaussian noise to each observation dimension. The perturbation magnitude is parameterized by $\sigma=2$, such that the noise is drawn from $\mathcal{N}\left(0, \sigma^{2}\right)$. Each value of $\sigma$ defines a distinct perturbation phase, which we treat as a separate task in the continual learning sequence.

## Results

To quantify differences in performances we calculate the rewards that methods accumulated across the whole simulation. Figure 1 lists the cumulative sum of mean episode reward on Gym Control environments. We can see that the modular topology achieves the best results for CartPole and Lunarlander. In Acrobot, the differences between the networks are only marginal, as can also be seen visually in Figure 1. In both Cartpole and Lunarlander, the modular topology achieves a cumulative reward that is almost twice as high as the MLP baseline. The policy collapse of the MLP is most evident in the CartPole task. A comparison between hybrid and small-world shows that both exhibit more plasticity than the baseline, but do not match the performance of
the modular network. Figure 1 allows direct observation of the loss of plasticity after the introduction of the first distribution shift.

Table 1: Cumulative Rewards during training across network topologies, scaled by $10^{5}$ across three control tasks.
| Topology | CartPole | Acrobot | Lunarlander |
| :--- | :---: | :---: | :---: |
| Modular | $2.18 \pm 0.5$ | $-1.14 \pm 0.2$ | $-7.41 \pm 0.2$ |
| Hybrid | $1.66 \pm 0.5$ | $-1.04 \pm 0.6$ | $-10.0 \pm 0.4$ |
| Small-World | $1.27 \pm 0.8$ | $-1.07 \pm 0.8$ | $-8.9 \pm 0.5$ |
| MLP | $1.27 \pm 0.09$ | $-1.12 \pm 0.1$ | $-12.5 \pm 0.8$ |


![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-319.jpg?height=656&width=650&top_left_y=813&top_left_x=1169)
Figure 1: Overall comparison in continual control. We consider three simple control tasks where random noise is added to the observations and is being resampled every 200 Iterations (grey vertical lines).

## Discussion

For Cartpole and Lunarlander, the modular network topology outperforms the MLP baseline and the networks with small-world characteristics. Further investigation is necessary to analyse what happens inside the networks, and if functional modularity emerges besides the predefined structural modularity. Such seperation might be leading to increased robustness against plasticity loss. Although the results show only marginal differences between performances on Acrobot, the differences during training imply that network topology influences performance in continual RL.

Due to time and resource constraints, the experiments were only performed on the slightly out-dated Gym Control RL tasks and with few parameter variations. To better evaluate the potential of additional parameter exploration in network topologies, they would need to be applied to state-ofthe Art Benchmarks like Continual World (Wołczyk et al., 2021).

## Acknowledgements

Funded by the European Union (ERC, GROW-AI, 101045094). Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council.

## References

Abel, D., Barreto, A., Van Roy, B., Precup, D., van Hasselt, H. P., and Singh, S. (2023). A definition of continual reinforcement learning. Advances in Neural Information Processing Systems, 36:50377-50407.

Albert, R. and Barabási, A.-L. (2002). Statistical mechanics of complex networks. Reviews of modern physics, 74(1):47.

Chaganty, A. T., Gaur, P., and Ravindran, B. (2012). Learning in a small world. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent SystemsVolume 1, pages 391-397.

Early, J. (2019). Reducing catastrophic forgetting when evolving neural networks. arXiv preprint arXiv:1904.03178.

Farebrother, J., Machado, M. C., and Bowling, M. (2018). Generalization and regularization in dqn. arXiv preprint arXiv:1810.00123.

Hintze, A. and Adami, C. (2008). Evolution of complex modular biological networks. PLoS computational biology, 4(2):e23.

Khetarpal, K., Riemer, M., Rish, I., and Precup, D. (2022). Towards continual reinforcement learning: A review and perspectives. Journal of Artificial Intelligence Research, 75:1401-1476.

McAllister, J., Wade, J., and O'Donnell, C. (2022). Heterosynaptic plasticity rules induce small-world networks.

Mirzadeh, S. I., Chaudhry, A., Yin, D., Nguyen, T., Pascanu, R., Gorur, D., and Farajtabar, M. (2022). Architecture matters in continual learning. arXiv preprint arXiv:2202.00275.

Muppidi, A., Zhang, Z., and Yang, H. (2024). Fast trac: A parameter-free optimizer for lifelong reinforcement learning. Advances in Neural Information Processing Systems, 37:51169-51195.

Pan, W., Zhao, F., Han, B., Dong, Y., and Zeng, Y. (2024). Emergence of brain-inspired small-world spiking neural network through neuroevolution. Iscience, 27(2).

Shelton, D. S. and Martins, E. P. (2017). Behavioral variation, adaptation, and evolution.

Sugimoto, Y. A., Yadohisa, H., and Abe, M. S. (2025). Network structure influences self-organized criticality in neural networks with dynamical synapses. Frontiers in Systems Neuroscience, 19:1590743.

Wang, L., Zhang, X., Su, H., and Zhu, J. (2024). A comprehensive survey of continual learning: Theory, method and application. IEEE transactions on pattern analysis and machine intelligence, 46(8):5362-5383.

Wołczyk, M., Zajac, M., Pascanu, R., Kuciński, Ł., and Miłoś, P. (2021). Continual world: A robotic benchmark for continual reinforcement learning. Advances in Neural Information Processing Systems, 34:28496-28510.

# Strategy Extraction from Reinforcement Learning via LLM-Guided Natural Language Annotation 

Ilya Horiguchi ${ }^{1}$, Michael Crosscombe ${ }^{1}$, Ryosuke Takata ${ }^{1}$, and Takashi Ikegami ${ }^{1}$<br>${ }^{1}$ Department of General Systems Studies, Graduate School of Arts and Sciences, University of Tokyo, Japan<br>ilya-horiguchi@g.ecc.u-tokyo.ac.jp


#### Abstract

Deep reinforcement learning (RL) agents can outperform humans in complex tasks, yet their learned policies remain opaque. We present a pipeline that converts the implicit knowledge of a high-performing RL agent into an explicit, human-readable strategy library using a large language model (LLM). First, a teacher agent is trained with Proximal Policy Optimisation in the partially observable game Connect Four. The agent's gameplay trajectories are then automatically annotated by an LLM in a structured LanguagePolicy Markup Language (LPML) containing <Condition>, <Thought>, and <Execution> tags. Unsupervised clustering of the resulting annotations yields a compact library of reusable tactics.


## Submission type: Late Breaking Abstract

Code available at: https://anonymous.4open. science/r/SEGP-D819

## Introduction

Deep reinforcement learning (RL) now achieves superhuman performance in domains from Atari to Go, but its decision processes remain opaque-hindering use in safetycritical or design-heavy settings. Recent surveys note a persistent trade-off between transparency and sample efficiency: interpretable pipelines often require costly post-hoc probes or sacrifice performance for simplicity Milani et al. (2023); Cheng et al. (2025).

Large language models (LLMs) suggest a path forward. Prior work uses LLMs to propose sub-goals (ELLM), perform self-critique (Reflexion), insert concept bottlenecks, or pair language with environment models Du et al. (2023); Shinn et al. (2023); Delfosse et al. (2024); Zhong et al. (2022). However, these either treat the LLM as an auxiliary shaper or entangle explanation with control, leaving open how to extract, compress, and reuse high-level strategies in an editable form.

We introduce a two-stage pipeline: (i) annotate a high-capacity teacher policy in a Language-based Policy Mark-up Language (LPML; Yamauchi et al., 2023) via

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-322.jpg?height=250&width=842&top_left_y=740&top_left_x=1107)
Figure 1: A conceptual LPML-based agent loop. Agents iteratively describe the current state (<Condition>), reason in natural language (<Thought>), and issue executable commands (<Execution>).

ChatGPT, yielding explicit <CONDITION, THOUGHT, EXECUTION> triples; (ii) cluster these free-form annotations to build a compact tactic library that transfers to other agents, including LLM-based players.

We run a multi-agent simulation where each decision cycle is expressed in LPML: agents emit <Condition, Thought, Execution>, the command is parsed and executed, and beliefs are updated before the next epoch (Figure 1). This loop enables transparent reasoning, sharing, and refinement of strategies, and supports the study of emergent tactics and social norms Horiguchi et al. (2024).

## Related Work

Explainable RL (XRL) includes saliency methods, rule extraction, and concept bottlenecks; surveys by Milani et al. and Cheng et al. compare these approaches and highlight the rising role of LLMs and RLHF Milani et al. (2023); Cheng et al. (2025). Concept-based lines such as SCoBots enforce successive human-aligned bottlenecks to mitigate reward hacking Delfosse et al. (2024), while HEX-RL integrates think-aloud natural-language rationales directly into policies Peng et al. (2022). In parallel, LLMs have been used primarily to shape rather than explain RL: ELLM supplies sub-goal rewards Du et al. (2023), DYSTIL queries online strategy sketches Wang et al. (2025), and Reflexion performs verbal self-evaluation as an outer-loop optimiser Shinn et al. (2023). Related distillation work compresses teacher policies without interpretability Rusu et al. (2015), augments
![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-323.jpg?height=433&width=858&top_left_y=187&top_left_x=173)
training with structured advice Watkins et al. (2021), or cotrains language to describe dynamics Zhong et al. (2022). Our approach differs by harvesting language post hoc from a teacher, clustering it into a compact tactic library, and enabling editing and reuse without retraining.

## Approach \& Evaluation

Setup. We study Connect Four ( $6 \times 7$; gym-connect 4) and convert a high-capacity PPO teacher into a compact, interpretable tactic library via LPML annotation and clustering. The teacher is a 2-layer ( 256,256 , ReLU) MLP actor-critic trained for $5 \times 10^{4}$ steps against a hand-crafted opponent curriculum with periodic selfplay, achieving $>95 \%$ win rate vs. the strongest builtin agent. We record $N_{\text {traj }}=10$ deterministic games (greedy policy) vs. BABY, storing $\left(s_{t}, a_{t}, r_{t}\right.$, done $\left._{t}\right)$. For each $\left(s_{t}, a_{t}\right)$, an 03-2025-04-16 model (5-shot) emits an LPML quadruple <Condition, Thought, Execution, Action> with a self-consistency filter (reject/regenerate $<3 \%$ ). We embed Condition+Thought (all-MiniLM-L6-v2) and agglomeratively cluster (cosine, cutoff $d=0.3$ ) into $K=7$ tactics covering $>90 \%$ of decisions; medoids are summarised to form $\mathcal{L}$.

Agents and Retrieval. Two gpt- $4 \circ-$ mini agents (temperature 0.3) are compared: a control (no tactics) and an LPML-enhanced agent that receives top- $k=3$ tactics retrieved from $\mathcal{L}$ (ChromaDB, cosine) given the current state.

Protocols. (i) 40 head-to-head games measuring win rate; (ii) 10 games each ( 5 first/ 5 second) vs. rule-based Baby/Child/Teenager/Adult; (iii) a one-game case study vs. Baby tracing retrieved tactics and similarity scores.

## Results

Using the gpt-4o-mini agents (temperature 0.3) with top- $k=3$ tactic retrieval, the LPML-enhanced agent won 28/40 head-to-head games against the control ( 0.70 vs. 0.30 win rate), and the average score shifted from -0.40 (control) to +0.40 (enhanced), a $133 \%$ relative win-rate gain. These outcomes are summarised in Fig. 2a. Against rule-based opponents ( 10 games per agent and role), the enhanced
agent consistently outperformed the control-Child 0.60 vs. $0.45(+0.15)$, Teenager 0.05 vs. 0.00 , Adult 0.05 vs. 0.00-with the trend visible across the skill gradient in Fig. 2b. A qualitative case study (Fig. 3) shows the enhanced agent using retrieved tactics critically rather than blindly: despite occasional irrelevant snippets, it overrode them and executed the decisive vertical four by playing column 3 on Turn 17.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-323.jpg?height=515&width=558&top_left_y=596&top_left_x=1254)
Figure 3: Board at Turn 17 vs. BabyPlayer. The LPMLenhanced agent (' X ') wins with its ninth move in column 3 (vertical four), having disregarded irrelevant retrieved snippets.

## Conclusion

We proposed a domain-agnostic pipeline that turns an RL teacher's gameplay into a compact, interpretable LPML tactic library via annotation and clustering. Used as retrieval guidance (top- $k=3$ ) for a gpt- $4 \circ-\operatorname{mini}$ agent (temperature 0.3) in Connect Four, the library improved control: the LPML-enhanced agent won 28/40 head-to-head games over a vanilla LLM and showed consistent, if modest, gains versus rule-based opponents. A case study showed tactics are integrated critically, with the agent overriding irrelevant snippets to execute a winning vertical connect, supporting both competence and auditability through readable, editable strategies.

Limitations: a single deterministic domain with small samples and no CIs; occasional biased/irrelevant retrieval; conservative baselines; missing ablations (cutoff, $k$ ); and unreported annotation cost. Future work: add statistical rigor, stronger baselines, and multi-domain tests, and extend to language-level self-play via evolutionary search-treating LPML tactics as "genomes" evolved by tournament selection, crossover of <Condition, Execution>, and mutation. This fuses RL interpretability with natural-language evolution to surpass the teacher, reduce bias, and yield auditable behavior distilled into student policies.

## Acknowledgements

This work was supported by JSPS KAKENHI Grant-in-Aid for Scientific Research (A) (24H00707), "Community-First Theory: A Theory and Experiments on the Evolution of Individuality, Diversity, and Spontaneity." Ilya Horiguchi was supported by a Grant-in-Aid from the Foundation for Fusion of Science and Technology (FOST).

## References

Cheng, Z., Yu, J., and Xing, X. (2025). A survey on explainable deep reinforcement learning. arXiv preprint arXiv:2502.06869.

Delfosse, Q., Sztwiertnia, S., Rothermel, M., Stammer, W., and Kersting, K. (2024). Interpretable concept bottlenecks to align reinforcement learning agents. In Advances in Neural Information Processing Systems.

Du, Y., Watkins, O., Wang, Z., Colas, C., Darrell, T., Abbeel, P., Gupta, A., and Andreas, J. (2023). Guiding pretraining in reinforcement learning with large language models. In Proceedings of the 40th International Conference on Machine Learning.

Horiguchi, I., Yoshida, T., and Ikegami, T. (2024). Evolution of social norms in llm agents using natural language.

Milani, S., Topin, N., Veloso, M., and Fang, F. (2023). Explainable reinforcement learning: A survey and comparative review. ACM Computing Surveys.
Peng, X., Xing, C., Choubey, P. K., Wu, C., and Xiong, C. (2022). Inherently explainable reinforcement learning in natural language. In Advances in Neural Information Processing Systems.

Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., Mnih, V., Kavukcuoglu, K., and Hadsell, R. (2015). Policy distillation. arXiv preprint arXiv:1511.06295.

Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K. R., and Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366.

Wang, B., McKeown, K., and Ying, R. (2025). DYSTIL: Dynamic strategy induction with large language models for reinforcement learning. arXiv preprint arXiv:2505.03209.

Watkins, O., Gupta, A., Darrell, T., Abbeel, P., and Andreas, J. (2021). Teachable reinforcement learning via advice distillation. In Advances in Neural Information Processing Systems.

Yamauchi, R., Sonoda, S., Sannai, A., and Kumagai, W. (2023). Lpml: Llm-prompting markup language for mathematical reasoning.

Zhong, V., Mu, J., Zettlemoyer, L., Grefenstette, E., and Rocktäschel, T. (2022). Improving policy learning via language dynamics distillation. In Advances in Neural Information Processing Systems.

# Lowering Collective Free Energy for Consensus in LLM Agent Collectives: Common Ground via Collective Predictive Coding 

Yusuke Hayashi ${ }^{1}$, Kazuya Horibe ${ }^{2}$, and Kotaro Sakamoto ${ }^{3}$<br>${ }^{1}$ AI Alignment Network, Japan, hayashi@aialign.net<br>${ }^{2}$ RIKEN, Japan<br>${ }^{3}$ University of Tokyo, Japan


#### Abstract

LLM agent collectives are now embedded in many services, making it increasingly important for AI itself to support consensus formation and the discovery of common ground. From an Artificial Life (ALIFE) perspective, whether synthetic populations can discover and stabilize common ground determines if emergent communication integrates or collapses. We ground mediation in Collective Predictive Coding (CPC), which characterizes consensus as a regime in which each agent's internal model aligns with a shared external representation while the population's collective free energy de-creases-at which point agents' Bayesian networks effectively couple and a communication channel opens. We instantiate this perspective in a controlled simulation that contrasts a CPC mediator with a purely aggregative baseline. The CPC mediator selects shared statements that are informative about the population's current state and gently steers agents toward them. Across deliberation rounds, this produces a steady increase in shared understanding and a contraction of within-group dispersion while maintaining comparable acceptance of proposals. In other words, groups converge faster and more coherently without sacrificing endorsement. These findings motivate an energy-aware, principled design rule for AI mediation in LLM societies: select messages that explain the group's present state and update internal representations in the direction that lowers collective free energy. ALIFE offers an ideal testbed for such mechanisms, linking emergent communication with normative control of collective dynamics and paving the way for human-subject validation.


Submission type: Late Breaking Abstracts

## Introduction \& Related Work

Why common ground matters to ALIFE. Modern AI services increasingly deploy multi-agent LLM systems that negotiate, plan, and fact-check collaboratively. From an ALIFE perspective, these systems are synthetic populations in which communication, symbol emergence, and collective intelligence arise from local interactions. Whether such populations discover common ground is decisive for their stability and usefulness, aligning with ALIFE's long-standing interest in emergent communication and information encoding in life-like systems.

From aggregation to principled mediation. Recent work such as the Habermas Machine (HM) shows that LLMs can synthesize common-ground statements at scale (Tessler et al., 2024). However, purely aggregative pipelines provide limited guidance about what makes a proposal normatively good for the collective. The CPC hypothesis fills this gap by grounding consensus in the collective free energy of a generative model over external symbols $w$, internal codes $\mathbf{z}$, and observations. At the stationary point, the population's Bayesian networks couple at the node $w$ so that a communication channel emerges and the ensemble behaves as a single effective network. This gives a mathematically explicit picture of consensus and a normative design rule: propose $w$ informative about current $z$ and gently regularize $z \rightarrow \mu_{w}$ to reduce collective free energy (Taniguchi, 2024; Taniguchi et al., 2025; Friston, 2010).

Our position and contributions. We bring this CPC view to AI mediation, testing the hypothesis that energyaware selection of $w$ and CPC-style $z$ updates improve consensus. Concretely, we compare a CPC mediator against a non-CPC baseline on simulated deliberation with five value dimensions. Our results show higher final alignment, lower dispersion, and larger estimated $\hat{\mathcal{I}}(w ; \mathbf{z})$ for CPC at comparable acceptance, suggesting that CPC provides a principled upgrade path from aggregative LLM mediation toward ALIFE-grounded mechanisms of emergent communication.

## Derivation: From CPC Free Energy to the Tempered Posterior

We sketch a derivation that makes the above choice of $q^{*}(w \mid$ z) principled under CPC's generative semantics.

CPC factorization. For $K$ agents with local observations $\mathbf{o}=\left\{o^{k}\right\}_{k}$ and latents $\mathbf{z}=\left\{z^{k}\right\}_{k}$, CPC assumes

$$
\begin{aligned}
& p_{\theta}(w, \mathbf{z}, \mathbf{o})=p_{\theta}(w) \prod_{k=1}^{K} p_{\theta}\left(z^{k} \mid w\right) p_{\theta}\left(o^{k} \mid z^{k}\right), \\
& q_{\phi}(w, \mathbf{z}, \mathbf{o})=q_{\phi}(w \mid \mathbf{z}) \prod_{k=1}^{K} q_{\phi}\left(z^{k} \mid o^{k}\right) q_{\phi}\left(o^{k}\right) .
\end{aligned}
$$

Collective free energy decomposition. With this structure, the collective free energy splits as

$$
\mathcal{F}\left(q_{\phi}, p_{\theta}\right)=\mathcal{C}(w ; \mathbf{z})+\mathcal{R}(\mathbf{z} ; w, \mathbf{o})+\mathcal{D}(\mathbf{o} \mid \mathbf{z})
$$

Budgeted optimization and Lagrangian. Listeners minimize their individual surprise/regularization subject to a collective-regularization budget:

$$
\min _{q_{\phi}} \mathcal{R}(\mathbf{z} ; w, \mathbf{o})+\mathcal{D}(\mathbf{o} \mid \mathbf{z}) \text { s.t. } \mathcal{C}(w ; \mathbf{z}) \leq \rho \text {. }
$$

Introducing a multiplier $\lambda>0$ yields the Lagrangian

$$
\mathcal{J}\left(q_{\phi}, p_{\theta}\right)=\lambda(\mathcal{C}(w ; \mathbf{z})-\rho)+\mathcal{R}(\mathbf{z} ; w, \mathbf{o})+\mathcal{D}(\mathbf{o} \mid \mathbf{z}) .
$$

Stationarity and Gibbs-form posterior. Taking the functional derivative w.r.t. $q_{\phi}(w \mid \mathbf{z})$ and setting it to zero,

$$
\frac{\delta \mathcal{J}}{\delta q_{\phi}(w \mid \mathbf{z})}=0 \Rightarrow q_{\phi}^{*}(w \mid \mathbf{z}) \propto p_{\theta}(w)\left[p_{\theta}(\mathbf{z} \mid w)\right]^{\frac{1}{\lambda}} .
$$

Thus $\lambda$ plays the role of an inverse temperature that controls how selectively symbols $w$ concentrating probability mass are chosen to better explain $\mathbf{z}$.

MH acceptance as a listener-side likelihood ratio. Embedding $q^{*}$ in a Metropolis-Hastings (MH) naming-game (Taniguchi et al., 2023) move (speaker proposes $w^{\prime}$, listener tests replacement of its current $w$ ) yields the acceptance

$$
\begin{aligned}
\alpha_{\mathrm{acc}} & =\min _{w^{\prime}}\left(1, \frac{q_{\phi}^{*}\left(w^{\prime} \mid \mathbf{z}^{\mathrm{Sp}}\right)}{q_{\phi}^{*}\left(w \mid \mathbf{z}^{\mathrm{Sp}}\right)}\right), \\
& =\min _{w^{\prime}}\left(1,\left[\frac{p_{\theta}\left(\mathbf{z}^{\mathrm{Li}} \mid w^{\prime}\right)}{p_{\theta}\left(\mathbf{z}^{\mathrm{Li}} \mid w\right)}\right]^{\frac{1}{\lambda}}\right),
\end{aligned}
$$

i.e., a temperature-controlled log-likelihood ratio test on the listener's encoder/prior; nuisance terms cancel so acceptance is fully decentralized. In our simulation we use Eq. (6) for selection and a cosine-similarity threshold as a lightweight acceptance proxy; Eq. (8) clarifies its CPCconsistent counterpart.

## Experimental Setup

We compare a CPC mediator to a non-CPC benchmark using the python simulation. Agents: $K=18$; Rounds: $R=25$; Value dimensions: \{Safety, Equality, Cost Effectiveness, Transparency, Sustainability\}. Key hyperparameters (CPC / Benchmark): $\alpha_{\text {obs }}=0.30$ (utterance-driven $z$ update), $\beta_{\text {reg }}=0.35$ ( $\times 0.3$ for Benchmark), $\lambda=0.5, \sigma=0.25$, $M=16$ consensus candidates per round. Metrics logged per round: acceptance rate, mean $\cos \left\langle z^{k}, \mu_{w}\right\rangle$, within-group variance $\mathcal{V}(\mathbf{z})=\frac{1}{K} \sum_{k=1}^{K}\left\|z^{k}-\overline{\mathbf{z}}\right\|_{2}$, and approximate mutual information $\hat{\mathcal{I}}(w ; \mathbf{z})=\mathbb{E}_{z}\left[\mathcal{D}_{\mathrm{KL}}(p(w \mid \mathbf{z}) \| p(w))\right]$.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-327.jpg?height=1050&width=664&top_left_y=278&top_left_x=1162)
Figure 1: Round-wise metrics from the simulation: (top) approximate $\hat{\mathcal{I}}(w ; \mathbf{z})$, (bottom) within-group variance $\mathcal{V}(\mathbf{z})$. CPC dominates or matches the benchmark across metrics.

## Results

Across rounds, the CPC condition steadily increases the informativeness of shared statements about the population state while contracting within-group dispersion, indicating quicker integration without hurting proposal acceptance. The CPC mediator improves integration and informativeness without sacrificing endorsement. Figure 1 shows the main indicators aggregated over runs.

## Discussion \& Outlook

These findings support the CPC prediction: mediation that (i) proposes statements $w$ that are informative about current $\mathbf{z}$ and (ii) gently regularizes $\mathbf{z} \rightarrow \mu_{w}$ reduces collective free energy and leads to more integrated consensus. Limitations include the stylized text-to-vector mapping, a small set of value dimensions, and simulated rather than human groups. Next, we plan to (a) validate with human participants and pre-registered outcomes (consensus quality, fairness, stability), (b) compare against HM-style pipelines (Tessler et al., 2024), and (c) examine robustness to minority protection and asymmetric priors over $w$. Our results suggest that the energy-aware design of AI mediators can provide principled performance gains over purely aggregative approaches.

## References

Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11(2):127-138.

Taniguchi, T. (2024). Collective predictive coding hypothesis: symbol emergence as decentralized Bayesian inference. Frontiers in Robotics and AI, 11:1353870.

Taniguchi, T., Takagi, S., Otsuka, J., Hayashi, Y., and Hamada, H. T. (2025). Collective predictive coding as model of science: formalizing scientific activities towards generative science. Royal Society Open Science, 12(6):241678.

Taniguchi, T., Yoshida, Y., Matsui, Y., Nguyen, L. H., Taniguchi, A., and Hagiwara, Y. (2023). Emergent communication through metropolis-hastings naming game with deep generative models. Advanced Robotics, 37(19):1266-82.

Tessler, M. H., Bakker, M. A., Jarrett, D., Sheahan, H., Chadwick, M. J., Koster, R., Evans, G., CampbellGillingham, L., Collins, T., Parkes, D. C., Botvinick, M., and Summerfield, C. (2024). Ai can help humans find common ground in democratic deliberation. Science, 386(6719):eadq2852.

# What Makes Quantum Biology Quantum - and Is It Necessary For Life? 

Moritz Kriegleder ${ }^{1}$, Andrea Loettgers ${ }^{1}$ and Tarja Knuuttila ${ }^{1}$<br>${ }^{1}$ University of Vienna, Austria

Corresponding author: moritz.kriegleder@univie.ac.at


#### Abstract

Quantum biology raises questions about the different roles that physics and biology assign to the environment, prompting a reevaluation of how the environment is understood in biological systems influenced by quantum effects. We explore the potential manifestations of quantum entanglement as a nontrivial quantum effect in biology, a phenomenon that shifts the perception of the environment from a perturbation to a functional role. While physicists have generally sought to isolate the phenomena under investigation, biologists view the environment as essential to development and evolution. These different scientific perspectives lead to novel issues in model transfer from quantum physics to biology. We contrast early attempts to use concepts from quantum mechanics, such as complementarity, in biology with a recent model of photosynthesis that suggests quantum entanglement enhances energy efficiency. Different explanatory strategies highlight the interdisciplinary tensions in quantum biology. While empirical evidence for quantum effects in biology remains limited, these theoretical explorations reveal that the intersection of quantum physics and biology is reshaping core concepts such as entanglement in open systems and the role of the environment in biological processes.


Submission type: Late Breaking Abstract

## How quantum is biology?

Recently, quantum biology has gained traction by applying novel modeling strategies from quantum physics to the study of biological phenomena. From magnetic field sensing in birds to the mechanism of smell in humans to photosynthesis, physicists have found biological phenomena that appear to have mechanisms that do not fit classical physics explanations (Abbott et al., 2008; Marais et al., 2018; McFadden and Al-Khalili, 2018). To this day, there are different positions on how much of biology can be explained by physics. From strict reductionists, who aim to reduce biological laws to physics, to organicists, who consider living dynamics beyond the scope of physical explanations, there are many interpretations of the relationship between biology and fundamental physics (Domondon, 2006; Longo et al., 2012).

This relationship between quantum physics and biology, as well as the significance of quantum effects for living systems, has been discussed since the early days of quantum physics. (Schrödinger, 1992; Bohr, 1933; Stent, 1968; Delbrück, 1999; Kay, 1985). Scientists pursuing quantum biology are resuming this line of inquiry now that more advanced modeling and experimental tools are available from physics and biology research, including methods and results from quantum information theory.

## Necessity for quantum models of life

In this paper we discuss how modeling entanglement in the "noisy" environment of biology involves an expansion of the role and concept of the environment. We argue that this change is largely due to how scientists in quantum biology relate quantum effects to the biological environment in the construction and design of models and experiments.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-330.jpg?height=761&width=759&top_left_y=1535&top_left_x=1110)
Figure 1: Many faces of quantum biology. Trivial quantum effects describe chemical properties derived from quantum physics, such as the stability and shape of molecules, which are not unique to biology. Non-trivial quantum biology deals with novel effects that could explain biological processes such as photosynthesis.

Quantum foundations of life include speculative theories that link quantum processes to fundamental life dynamics such as evolution and cognition. Instrumentalist approaches use biological molecules in quantum physics experiments without directly discussing the implications for biology.

The case of photosynthesis shows how entanglement and coherence give us reasons to abandon the worldview of classical physics when we want to explain biological systems. Delbrück's search for unique biological laws of replication and mutation had inspired many physicists to apply quantum physics to processes characteristic of life. According to Delbrück's understanding of Bohr's complementarity principle, there is no observation that is independent of the state of the observer. If entanglement really plays an essential role in biology, we would have to acknowledge the non-separability of biological complexes for processes such as photosynthesis.

At all intersections of physics and biology, we see the reuse of mathematical models, and we can trace how conceptual ideas change meaning when applied to biology (Figure 1). The recent interest in thermodynamically open quantum systems also suggests a reciprocal influence of biology back into quantum physics. In addition to formal models, physicists have used model organisms to test general physical principles in biological systems that are as simple as possible. Bacteriophages, for example, allowed Delbrück to observe how genetic material can be altered, replicated, and passed on to subsequent generations. His pioneering work revealed the importance of random mutations in bacteriophages, challenging deterministic notions of genetic evolution.

More recently, the study of quantum effects in photosynthesis has addressed the role of coherence and superposition in optimizing energy transfer. Because classical physical models have failed to account for efficient energy harvesting, quantum coherence and entanglement have been proposed as possible explanations for the remarkable efficiency observed in photosynthetic systems.

Philosophers of biology have debated what constitutes an environment in biological systems and how, especially in the case of evolutionary biology, organism and environment are decoupled, when perhaps an embedded view should be considered (Walsh, 2012, 2015). Modelers must decide how to integrate the environment in a way that captures the characteristic functions of the processes being modeled and ensures that the research problems remain tractable. As we discuss here, quantum biology not only leads to a rethinking of the scope of quantum physics, but also provides an interesting case for addressing the role of the environment in biology (Kabasenche et al., 2012).

The application of physics concepts to biology is far from trivial, because biological systems are outcomes of
evolution, complex, and thermodynamically open systems. The application of general theoretical frameworks from physics to biology is bound to reconfigure the meaning of physics concepts within biology, possibly feeding back into new theoretical work in different fields such as cognitive and social sciences. In this way, model transfer could not only reshape our understanding of the processes of life, but also initiate a transformative dialogue in which the boundaries between physics and biology become blurred, broadening the horizons of both fields.

## Acknowledgements

This project received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (Grant agreement No 818772).

## References

D. Abbott, P. C. W. Davies, and A. K. Pati, editors. Quantum aspects of life. Imperial College Press ; Distributed by World Scientific, London : Hackensack, NJ ; London, 2008.
M. Delbrück. A physicist looks at biology. Resonance, 4(11):89-102, Nov. 1999.
A. T. Domondon. Bringing physics to bear on the phenomenon of life: the divergent positions of Bohr, Delbrück, and Schrödinger. Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences, 37(3):433-458, Sept. 2006.
L. E. Kay. Conceptual models and analytical tools: The biology of physicist Max Delbr" uck. Journal of the History of Biology, 18(2):207-246, June 1985.
W. P. Kabasenche, M. O'Rourke, and M. H. Slater, editors. The Environment: Philosophy, Science, and Ethics. MIT Press, 2012.
G. Longo, M. Montevil, and S. Kauffman. No entailing laws, but enablement in the evolution of the biosphere. In Proceedings of the 14th annual conference companion on Genetic and evolutionary computation, GECCO '12, pages 1379-1392, New York, NY, USA, July 2012.
A. Marais, B. Adams, A. K. Ringsmuth, M. Ferretti, J. M. Gruber, R. Hendrikx, M. Schuld, S. L. Smith, I. Sinayskiy, T. P. J. Krüger, F. Petruccione, and R. van Grondelle. The future of quantum biology. Journal of The Royal Society Interface, 15(148):20180640, Nov. 2018.
J. McFadden and J. Al-Khalili. The origins of quantum biology. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 474(2220):20180674, Dec. 2018.
E. Schrödinger. What is Life?: With Mind and Matter and Autobiographical Sketches. Cambridge University Press, Jan. 1992.
G. S. Stent. That Was the Molecular Biology That Was. Science, New Series, 160(3826):390-395, 1968.
D. Walsh. Mechanism and purpose: A case for natural teleology. Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences, 43(1):173-181, Mar. 2012.
D. M. Walsh. Organisms, Agency, and Evolution. Cambridge University Press, 1 edition, Nov. 2015.

# Communication Matters: LLM Simulations of Cooperation in El Farol Bar Problem 

Ryosuke Asada1, Keita Nishimoto1, Kimitaka Asatani1, Ichiro Sakata1<br>${ }^{1}$ University of Tokyo, Japan<br>asada3229@g.ecc.u-tokyo.ac.jp


#### Abstract

This study explores how conversational channel design shapes cooperation in the El Farol Bar problem using large language models (LLMs). We compared open chat, pairwise direct messages, and restricted direct messages across 50 games of 50 rounds each. Open chat increased failure and score variability, while direct messaging produced more stable two-player coordination. Score distributions showed frequent "one winner" or "one loser" outcomes, with balanced high scores rare. Players repeated prior choices $\sim 80 \%$, and conversation styles differed by setting, underscoring communication design's role in multi-agent cooperation.


type: Late Breaking Abstract

## Introduction

Since GPT-4 and subsequent large language models (LLMs), task-oriented agents and frameworks such as Microsoft AutoGen have made LLM-LLM and LLMhuman collaboration increasingly realistic[1]. At the same time, recent studies-using settings such as repeated Prisoner's Dilemma or Diplomacy-like games-report that LLMs may prioritize self-interest over cooperation, and even engage in deception, highlighting the importance of their social behavior [2][3]. In this context, I focus on the role of conversation channels. In social psychology and experimental game theory, it is well established that cooperation levels vary under peer monitoring, and that asymmetries or secrecy of information shape human interaction in real society. By analogy, I hypothesize that the design of conversational channels may also affect the interactions, cooperative tendencies, and sociality of LLMs, and we aim to examine these causal effects.

## Conducting the Simulation

In this study, we conducted a simulation of the El Farol Bar problem[4], a well-known framework in experimental economics, with large language models (LLMs) serving as the players. The experiment involved three players, and their payoffs varied according to the decisions made in each round. The procedure within a single round unfolded as follows:
(1) Strategy formation - Each player reflected on past outcomes and accumulated scores in order to consider a strategy for the current round.
(2) Discussion phase - Players engaged in free-form conversation regarding prior decisions and contextual factors from earlier rounds.
(3) Decision-making - Based on both strategic considerations and the conversational exchange, each player chose either GO (attend the bar) or STAY (stay at home), providing a justification for their choice.

The payoff rule was defined as follows: if at most two players selected GO, each of them received +2 points; if all three players selected GO, each incurred a -2 point penalty; and if a player selected STAY, their score remained unchanged. We implemented the model using gpt-4o-mini, accessed via API and orchestrated through LangChain, which provided the rules, past results, and contextual prompts to the model.

A central focus of this simulation was the structure of conversational channels among the players. Drawing from the perspectives of information asymmetry and secrecy, we designed three experimental conditions:
(A) Open Chat - All three players engaged in a shared, transparent conversation.
(B) Pairwise Direct Messages - Each pair of players communicated privately, with no global visibility.
(C) Restricted Direct Message - Only a designated pair of players exchanged private messages, while the third remained excluded.

## Result

For the purposes of the experiment, a single game was defined as 50 consecutive rounds. To ensure robustness of the findings, we conducted 50 independent games under each of the three conversational channel conditions. The figure above categorizes the state of each round within a game according to the number of players who chose GO, and then presents the average distribution of states across the 50 rounds of a game under each conversational setting.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-335.jpg?height=378&width=698&top_left_y=351&top_left_x=203)
Fig1 : The average distribution of states in a game

In the case of (A) Open Chat, however, the probability that all three players selected GO-which results in collective failure-was notably higher, leading to greater variability in score outcomes. By contrast, under the (B) Pairwise Direct Message and (C) Restricted Direct Message conditions, the proportion of rounds in which exactly two players chose GO was consistently higher, suggesting a more stable coordination pattern.

| mode | $\max$ | $\min$ | average |
| :---: | :---: | :---: | :---: |
| (A) | 13.3 | -20.8 | -1.6 |
| (B) | 39.4 | -9.5 | 17.6 |
| (C) | 56.2 | 0.0 | 32.1 |

Table1 : The averages of scores

The table above reports the average of the highest score, lowest score, and mean score achieved by players across games under each conversational condition. As the results clearly indicate, the outcomes vary substantially depending on the structure of communication. In particular, when all players engaged in an open group discussion (Condition A), overall performance was markedly lower, with both the maximum and average scores being depressed relative to the direct-message settings. This suggests that open, fully transparent communication may increase the likelihood of collective failure, thereby reducing the efficiency of coordination.

Next, we categorized the distribution of scores among the three players in each game into four distinct patterns, focusing on whether outcomes reflected balanced performance or clear disparities:
$\alpha$ : All Close - When the difference between the maximum and minimum scores was within $30 \%$ of the absolute maximum score, indicating that all players performed at roughly the same level. $\beta$ : One Loser - When the lowestscoring player lagged significantly behind the other two: specifically, if the gap between the smallest and the middle value was at least twice as large as the gap between the
middle and the highest value, while the latter gap remained within $30 \%$ of the maximum score. $\gamma$ : One Winner When the highest-scoring player clearly outperformed the others: specifically, if the gap between the largest and the middle value was at least twice as large as the gap between the middle and the lowest value, while the latter gap remained within $30 \%$ of the maximum score. $\delta$ : Dispersed - When none of the above conditions were met, and the distribution showed no clear balance or single

The classification results are presented below.

![](https://cdn.mathpix.com/cropped/fed660c6-4d3d-4bb6-bb3c-f9b039187660-335.jpg?height=369&width=791&top_left_y=813&top_left_x=1087)
Fig2 : The classification of the distribution of score

Across all conversational settings, the most common outcomes were two winners or dispersed scores. An intriguing finding emerged in the restricted direct message condition: in most cases, the two winners were composed of one member of the communicating pair together with the excluded player. From the perspective of cooperation, the ideal outcome would have been that all players achieved similarly high scores, yet such balanced results were rarely observed.

We also observed that players exhibited a strong tendency to repeat their previous choices: the probability of an individual making the same decision as in the prior round was close to $80 \%$, and consequently, the overall game state often persisted across rounds.

Regarding the conversational content, clear differences emerged across settings. In the open-chat condition, discussions were largely abstract, focusing on general considerations or risk awareness. In contrast, in the pairwise direct-message condition, conversations more frequently involved concrete proposals and requests, encouraging specific actions. Finally, under the restricted direct-message condition, where only two players communicated and the third was unaware of the exchange, the dialogue frequently included explicit references to the absent player's potential decisions.

## Reference

[1] Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A. H.,

White, R. W., Burger, D., \& Wang, C. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. arXiv preprint.
[2] Akata, E., Schulz, L., Coda-Forno, J., Oh, S. J., Bethge, M., \& Schulz, E. (2025). Playing repeated games with Large Language Models. Nature Human Behaviour
[3] Taylor, S. M., \& Bergen, B. K. (2025). Do Large Language Models Exhibit Spontaneous Rational Deception? arXiv preprint.
[4] Arthur, W. Brian. "Inductive reasoning and bounded rationality." The American economic review 84.2 (1994): 406-411.


[^0]:    ${ }^{1}$ https://plato.stanford.edu/entries/panpsychism/

[^1]:    ${ }^{1}$ For example, $k=1$ shows a friend relationship, $k=2$ shows a friend of friend.

[^2]:    ${ }^{1}$ See:Meta Quest Passthrough API

[^3]:    ${ }^{1}$ By simple rules, we mean interaction mechanisms with low descriptional complexity relative to the complexity of the resulting emergent behavior.
    ${ }^{2}$ While locality of interactions is not strictly required in a complex system, every self-organizing system observed in nature relies on local interactions among its components.

[^4]:    ${ }^{3}$ Biologically analogous to early gene-expression gradients which provide positional cues for subsequent formations of tissues and organs.

[^5]:    ${ }^{1}$ Exact algorithmic details of the method and the choice of hyperparameters are provided in the documentation TODO.
    ${ }^{2}$ https://lenia-explorer.vercel.app/

[^6]:    ${ }^{1}$ https://littlealchemy2.com/

[^7]:    ${ }^{1}$ https://github.com/Nogarx/Spark

[^8]:    ${ }^{1}$ Code: github.com/IteratedLM/2025_05_7Seg

[^9]:    ${ }^{1}$ In this work, we use life specifically to refer to an agent (used strictly in the "entity" sense of the word agent) traveling from one inhabited planet to a target non-inhabited planet, and whose arrival at the target planet triggers habitation.
    ${ }^{2}$ Because our detection method is based on overarching effect that life has on other planets as it spreads out across interstellar space, we cannot identify planets which are inhabited, yet incapable of interstellar travel. Thus, our use of the word inhabited implies two properties: inhabited and capable of interstellar travel.
    ${ }^{3}$ The approach described here is most similar to directed panspermia by intelligent life, but could also be envisioned as undirected lithopanspermia, where we refrain from burdening the simulations with agents that won't ever be able to interact with planets under the rules of our model. Ultimately, this has implications for the timescales implied in our model which will be explored in more

[^10]:    detail as we run more analyses under different conditions of stellar dynamics.

[^11]:    ${ }^{1}$ https://www.minecraft.net
    ${ }^{2}$ https://www.spigotmc.org/
    ${ }^{3}$ https://pyspigot-docs.magicmq.dev/

[^12]:    *Corresponding author: vassilis.physics@gmail.com

[^13]:    ${ }^{1}$ Usually, by 'preference' we will mean 'interestingness'. But any metric which is easy to judge for humans can be used e.g., 'amount of movement', 'colorfulness', 'funnyness'...

[^14]:    ${ }^{2}$ One could train the Rewarder giving it directly the parameters, as they contain the same information, in principle. In practice, this is unlikely to work (and hasn't worked in our tests) because it is hard to predict what dynamics will look like given only the parameters, as the relationship is highly non-linear and often chaotic.

[^15]:    1Experiment videos are available on the companion website.

[^16]:    ${ }^{1}$ Code is available at https://github.com/smearle/pathfindingnca

[^17]:    *Denotes equal contribution.

[^18]:    ${ }^{1}$ https://pogobot.github.io/

[^19]:    ${ }^{1}$ https://www.uiua.org/

