Humans Inside: Toward Digital Twins with Simulated Human Behavior
By Antti Oulasvirta
Abstract
The rapid advance of computing technology presents both an opportunity and a challenge for society. At the heart of this challenge lies a simple but difficult technical problem: predicting, counterfactually, how a novel design might change user behavior and experience. In this talk, I begin by discussing why previous approaches—including the prevailing human-centered design process as well as cognitive models—have fallen short. I then argue that recent advances at the intersection of software simulation, machine learning, and cognitive science are making it possible to bridge this gap. Emerging software simulators now provide richly detailed replicas of users’ real-world environments. Training and testing computational models of users within these simulators enables researchers to account for a wider range of factors, thereby producing more relevant predictions, inferences, and interventions—while remaining grounded in theory. This capability is opening a new frontier for applying computational models to the engineering and design of computers that better serve their users. I will present examples of state-of-the-art work and conclude with a discussion of the technical challenges that lie ahead.
Bio
Antti Oulasvirta leads the Computational Behavior Lab at Aalto University. Prior to joining Aalto, he was a Senior Researcher at the Max Planck Institute for Informatics and the Cluster of Excellence on Multimodal Computing and Interaction at Saarland University. He received his doctorate in Cognitive Science from the University of Helsinki in 2006, after which he was a Fulbright Scholar at the School of Information in University of California-Berkeley in 2007-2008 and a Senior Researcher at Helsinki Institute for Information Technology HIIT in 2008-2011. He was awarded the ERC Starting Grant (2015-2020) for research on computational design of user interfaces and the ERC Advanced Grant (2024-2029) for studying computational models of human behavior. His work has been awarded the Best Paper Award and Best Paper Honorable Mention at CHI sixteen times between 2008 and 2025. In 2025, he was invited to the CHI Academy. He was a SICSA Distinguished Visiting Fellow in 2011 and in 2022.
Photo Credit: UDE