Anna Giron 1, Simon Ciranka 2,3, Eric Schulz 4, Wouter van den Bos 5, Azzurra Ruggeri 2,6, Björn Meder 2,7, Charley M. Wu 1,2 1 University of Tübingen, Tübingen, Germany 2 Max Planck Institute for Human Development, Berlin, Germany 3 Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, Germany 4 Max Planck Institute for Biological Cybernetics, Tübingen, Germany 5 University of Amsterdam, Amsterdam, The Netherlands 6 Technical University Munich, Munich, Germany 7 Health and Medical University Potsdam, Potsdam, Germany
Learning from past experiences helps orient the exploration of unknown environments. Yet how we learn and explore changes over the lifespan, corresponding to different stages of cognitive development and different lifespan-rational goals. In this work, we analyze data from n=281 participants between the ages of 5 and 55, to model age-related changes in exploration and generalization. We use a spatially-correlated multi-armed bandit task (Wu et al,. 2018) that allows us to simultaneously model the separate and recoverable contributions of reward generalization (i.e., predicting novel outcomes), and both uncertainty-directed and random exploration. Through a combination of behavioral and model-based analyses, we show that exploration becomes more efficient over age, with increases in generalization, while both directed and random exploration decrease sharply in childhood. Counterfactual simulations also reveal that participants’ parameters move towards and reach the optimal frontier of learning strategies in adulthood. Furthermore, we compare the age-related trajectory of human parameter estimates to a stochastic optimization algorithm, as a rational comparison to how the parameters of our model (generalization, directed exploration, and random exploration) should optimally develop over time. These results reveal that people generalize less and exhibit less uncertainty-directed exploration compared to the optimal trajectory. Since both generalization and directed exploration are computationally costly, people may develop in a resource-rational manner, trading off performance against computational costs. Our work provides important insights into the developmental trajectory of human learning, providing a concrete empirical comparison to commonly used analogies of stochastic optimization in developmental psychology.
