Ben Chamberlain and James Rowbottom

Abstract

GRAND: Graph Neural Diffusion

We present Graph Neural Diffusion (GRAND) that approaches deep learning on graphs as a continuous diffusion process and treats Graph Neural Networks (GNNs) as discretisations of an underlying PDE. In our model, the layer structure and topology correspond to the discretisation choices of temporal and spatial operators. Our approach allows a principled development of a broad new class of GNNs that are able to address the common plights of graph learning models such as depth, oversmoothing, and bottlenecks. Key to the success of our models are stability with respect to perturbations in the data and this is addressed for both implicit and explicit discretisation schemes. We develop linear and nonlinear versions of GRAND, which achieve competitive results on many standard graph benchmarks.

https://arxiv.org/abs/2106.10934

Beltrami Flow and Neural Diffusion on Graphs

We propose a novel class of graph neural networks based on the discretised Beltrami flow, a non-Euclidean diffusion PDE. In our model, node features are supplemented with positional encodings derived from the graph topology and jointly evolved by the Beltrami flow, producing simultaneously continuous feature learning and topology evolution. The resulting model generalises many popular graph neural networks and achieves state-of-the-art results on several benchmarks.

https://arxiv.org/abs/2110.09443

Our speakers

Joint first authors of the papers James Rowbottom and Dr. Benjamin Paul Chamberlain. James received his master in AI with distinction from Imperial College London and worked on GRAND as an ML Research intern at Prof. Michael Bronstein's graph ML research group at Twitter. Ben is a Machine Learning Researcher at Twitter who received his PhD from Imperial College London.