About
First few hundred milliseconds: Reverse-engineering the brain’s algorithms for core vision
The visual system must not only recognize and localize objects, but also perform much richer inferences about the causes in the world underlying sense data. This research thrust aims to uncover the algorithmic basis of how we see so much so quickly, aiming to capture, in concrete engineering terms, something that we often take for granted: the breathtakingly fast and complex set of computations that occur in the brain between the moment we open our eyes and the moment a perceptual experience of a rich world appears in our minds.
Understanding human attention: Adaptive computation & goal-conditioned world models
Most scenes we encounter hold complex structure (e.g., in terms of objects, agents, events, places), but our goals render only a slice of this complexity relevant for perception. Attention somehow allows the mind to represent those objects in the world that are most relevant for our ongoing planning and actions. Whereas a rich tradition in experimental psychology has conceptualized attention in terms of objects and other structured mental representations, computational modeling work has almost uniformly formalized attention as a selective process determining what we sense, implemented as the weighting of feature embeddings with bottom-up (e.g., salience) or top-down factors (e.g., task templates). This research thrust aims to uncover the computational underpinning of how attention integrates goals to construct multi-granular, goal-conditioned world models.
Hypothesis-driven inquiry of how world models are implemented in neural populations
A central and shared challenge for cognitive science and neuroscience is to understand how in the brain the physical world is represented. Yet, the cognitive scientist and the neuroscientist do not necessarily speak the same language: a cognitive scientist might come to this challenge thinking about simulatable object representations and planning with them; the neuroscientist might come to it thinking about attractors and population dynamics. This research thrust is about developing new multi-level toolkits to enable faster and more significant discovery as to how the knowledge of the physical world might be implemented in the brain.
Intuitive physics basis of perception
Scene understanding is not just about recognizing what is where, but also seeing the physics of a scene. We can tell how heavy or elastic an object is by its feel when we hold it, but also by watching it move -- seeing it collide with other objects, bounce off the floor or splash into water. Even the shape of an object, a canonical aspect of core vision, is sometimes subject to the physics of a scene: The shape of a soft object is governed by its intrinsic dynamics and the external forces that apply on it. Similar issues arise for the perception of liquids, including its flow and viscosity. This research thrust develops novel computational models and objective, performance-based behavioral paradigms to study the "intuitive physics basis" of perception. We also test aspects of these models in fMRI experiments using animations of the world of physical objects.