Unifying Self-Supervised Clustering and Energy-Based Models

Abstract: Self-supervised learning has achieved remarkable progress in recent years due to its ability to extract high-quality representations from large amounts of unlabeled data. Meanwhile, generative models have offered valuable insights into the hidden processes that give rise to data. The synergy between these two areas of machine learning holds great promise, for example, by leveraging generative processes to learn more robust representations, or by using learned representations to synthesize new data.

However, a principled theory and methodology that bridge self-supervised learning and generative modeling are still lacking. In this work, we take a significant step toward closing this gap by demonstrating, for the first time, the feasibility of learning a self-supervised clustering model in a generative manner. To this end, we address three key challenges: formulation, integration, and unification.

Our empirical results show that this unification substantially improves the performance of existing self-supervised learners in clustering, data generation, and out-of-distribution detection. Moreover, we demonstrate that our approach can be incorporated into a neuro-symbolic framework to address a simple yet non-trivial instance of the symbol grounding problem, thereby enabling the learning of high-quality symbolic representations.

Bio: Emanuele Sansone is a Postdoctoral Fellow jointly affiliated with MIT (CSAIL) and KU Leuven (ESAT). His research interests lie at the intersection between unsupervised learning and mathematical logic. His research ambition is to empower machines with the capability to acquire and discover knowledge from data in an autonomous manner. He was recently awarded the Marie Curie Global Fellowship for the program titled “Discovering the World through Unsupervised Statistical Relational Learning”.

Zoom link: https://mit.zoom.us/j/99565154351