You are here

From pixels to semantics - machine learning as a key to understanding the dynamic computations along the human ventral stream

Dr. Tim Kietzmann
University of Cambridge, MRC Cognition and Brain Sciences Unit
Monday, November 26, 2018 - 11:30

The visual system is an intricate network of brain regions that enables us to recognise the world around us. To unravel the cortical mechanisms that underlie our ability to see, today’s neuroscience increasingly relies on machine learning to discover structures that lie latent in high-dimensional experimental data. Following this interdisciplinary approach, I here highlight a recent project in which we characterise and model the rapid representational dynamics across multiple stages of the human ventral stream using time-resolved brain imaging, multivariate pattern analyses, and deep learning. We observe substantial representational transformations during the first 300 ms of processing within and across ventral-stream regions. Categorical divisions emerge in sequence, cascading forward and in reverse across regions, and Granger causality analysis suggests bidirectional information flow between regions. To gain better insight into the complex dynamic computations that underlie these effects, we derived a novel approach for constraining large-scale deep neural network models with multivariate, time-resolved measurements of brain activity. Using this technique, we demonstrate that recurrent models outperform feedforward models in terms of their ability to jointly capture the multi-region cortical dynamics, suggesting that recurrent network architectures are required to understand information processing in the primate ventral stream. In addition to this case study, I will discuss our general work to better integrate deep learning into the computational neuroscience toolkit. For example, we recently introduced the largest, most ecologically valid dataset for training deep neural networks as models of human visual processing into the community. Moreover, we use tools from systems neuroscience to investigate the consistency and replicability of network-internal representations at different network depths. The insights derived from this project, as well as our hybrid approach that enables us to integrate neural data into the network weights, are expected to have direct implications beyond neuroscience, including artificial intelligence, robotics, and engineering applications.

Sign in