Seminar: Unifying Representations in Neural Models
| Speakers: Donato Crisostomi, Marco Fumero, PhD Students at Sapienza University | |
| Where: DIAG Department, Room A5 | |
| When: December 1st, 9:00 - 11:00 |
New findings in neuroscience and artificial intelligence reveal a shared pattern: whether in biological brains or artificial models, different learning systems tend to create similar representations when subject to similar stimuli.
In the first part of the talk, we’ll delve into why, how and when do different neural networks learn similar representations, and why this is useful. We’ll first remark some intriguing phenomena we have empirically observed, such as different latent spaces differing by an angle-preserving transformation. Then, we’ll see the reasons underlying the phenomenon and uncover techniques to make it happen. Finally, we’ll see measures to quantify representational similarity and touch on some possible applications.
In the second part of the talk, we will dive deeper in some of our recent works, showing when different neural models exposed to semantically related data learn intrinsically similar representations. In particular, we will show that their representational spaces can be connected to each other by means of simple transformations, which often take closed form solutions. This has significant implications for practical applications such as model stitching, reuse, and merging in various neural network architectures.
Speakers
Donato Crisostomi is a PhD student in the GLADIA research group at Sapienza, advised by prof. E. Rodolà . He graduated top of his class in Computer Science at the same university, winning the first-place in the respective honour programme. His experience encompasses a 3-months period as a PhD visitor at the University of Cambridge and two 6-months research internships at Amazon, investigating Knowledge Graphs in Luxembourg and Natural Language Understanding in Turin. Donato has published several works in top-tier AI journals and conferences such as ACM Surveys and ACL. He further served as program chair for the UniReps workshop at NeurIPS, delving in the same topics discussed in the talk. Beyond the academy, he is in the process of founding a no-profit organization to assist first-generation students in AI.
Marco Fumero is an ELLIS Ph.D. student at Sapienza University of Rome in the Gladia research group led by Professor Emanuele Rodolà . His previous experiences includes a research internship at Autodesk AI LAB and Amazon AWS AI working on geometric deep earning and causal representation learning topics. His research stands at the intersection of geometry and deep learning with a focus on representation learning, disentanglement and out-of-distribution generalization. He has been recently focusing on the direction of latent space communication, and, more broadly, on the question of when how and why distinct learning processes yield similar representation, organizing also a workshop at NeurIPS 2023 on these topics.