Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs

❤️ Become The AI Epiphany Patreon ❤️ https://www.patreon.com/theaiepiphany ‍‍‍ Join our Discord community ‍‍‍ https://discord.gg/peBrCpheKE In this video I cover "Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs" paper. The paper takes ideas from the sheaf theory - a branch of algebraic topology - and combines them with GNNs, enriching them with a rich geometric structure (sheaves) achieving provably more expressive diffusion-based graph neural networks! It took a lot of time to prepare this video and read everything that was necessary as a background reading - check out the resource section below for how to get started! ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ✅ Paper: https://arxiv.org/pdf/2202.04579.pdf Truly gentle introduction to sheaves: ✅ A Very Elementary Introduction to Sheaves: https://arxiv.org/pdf/2202.01379.pdf ✅ Opinion dynamics on discourse sheaves: https://arxiv.org/pdf/2005.12798.pdf ✅ Sheaf Neural Networks: https://arxiv.org/pdf/2012.06333.pdf Blogs: ✅ Blog accompanying the paper: https://towardsdatascience.com/neural-sheaf-diffusion-for-deep-learning-on-graphs-bfa200e6afa6 ✅ Differential geometry and algebraic topology papers overview: https://towardsdatascience.com/graph-neural-networks-through-the-lens-of-differential-geometry-and-algebraic-topology-3a7c3c22d5f ✅ Beginner intro to topology: https://www.cantorsparadise.com/what-is-topology-963ef4cc6365 ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ ⌚️ Timetable: 00:00:00 Gentle intro to sheaf theory and algebraic topology 00:12:00 Making it less abstract: examples from sheaf theory 00:23:50 Formal terminology 00:27:40 Sheaf Laplacian dissected 00:40:00 The separation power of sheaf diffusion 00:53:00 Dirichlet energy and converging to harmonic space 01:01:05 Neural Sheaf Diffusion GNN 01:06:00 Results and outro (feedback appreciated) ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ BECOME A PATREON OF THE AI EPIPHANY ❤️ If these videos, GitHub projects, and blogs help you, consider helping me out by supporting me on Patreon! The AI Epiphany - https://www.patreon.com/theaiepiphany One-time donation - https://www.paypal.com/paypalme/theaiepiphany Huge thank you to these AI Epiphany patreons: Eli Mahler Kevin Stone Petar Veličković ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ LinkedIn - https://www.linkedin.com/in/aleksagordic/ Twitter - https://twitter.com/gordic_aleksa ‍‍‍ Discord - https://discord.gg/peBrCpheKE YouTube - https://www.youtube.com/c/TheAIEpiphany/ Medium - https://gordicaleksa.medium.com/ GitHub - https://github.com/gordicaleksa AI Newsletter - https://aiepiphany.substack.com/ ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #sheaftheory #algebraictopology #oversmoothing #heterophily