Out-of-Distribution Generalisation, Predictive Uncertainty, and Continual Learning

Speaker: Mark van der Wilk, University of Oxford
Series: Computational Neuroscience

How can we create a learning algorithm that generalises “out of distribution”, i.e. beyond the training data? What can we expect about errors when a model generalises OoD? And what should happen when an error is made? In this talk, we will discuss how OccamLab’s approach to these questions. We will discuss solutions that are prescribed by a Bayesian perspective, through work on learning invariance/equivariance properties, uncertainty quantification, and continual learning. We will discuss how the Bayesian point of view points in the same direction as perspective from Minimum Description Length (MDL) and Kolmogorov Complexity, and finish with some open problems to work and collaborate on.

Bio: Mark van der Wilk is an Associate Professor at the University of Oxford. He is interested in whether learning algorithms can be improved by doing more than just fitting the training data. For example, whether neural network architectures can be adjusted automatically (instead of designed by humans), whether the data can tell which parts of a network must grow or can be shrunk, or what the causal interpretation of data should be. A unifying approach is that training should explicitly encourage compression of data (consistent Algorithmic Information Theory, or Bayesian Occam's Razor), instead of hoping that compression just emerges from fitting data. This idea is a thread that connects work on new learning algorithms, to Causal Foundation Models, and to applications in biochemistry.