Thesis:

Learning adversarial transformations via gradient inversion in a self supervised framework

In recent years, Self-Supervised Learning (SSL) has stood out as a valuable machine learning paradigm, allowing models to learn robust visual representations without relying on costly manual labeling. For visual data, non-contrastive SSL architectures learn representations by maximizing the similarity between representations of augmented views of the same image. Current state-of-the-art methods rely on heuristic image augmentations, the quality of which significantly impacts the final model performance and generalization capacity. We propose an adaptation that integrates adversarial learning into the SSL pipeline to learn adversarial spatial transformations. Our architecture incorporates a Spatial Transformer Network (STN) to perform differentiable image augmentations, paired with a Gradient Reversal Layer (GRL). The GRL enables the STN to be trained adversarially, maximizing the embedding-distance loss, thereby forcing the encoder to develop robust features and improving generalization. We evaluate our approach by incorporating the adversarial module into FastSiam, a state-of-the-art SSL architecture. Our proposed architecture outperforms the current state-of-the-art models on CIFAR10, converging to previous best evaluations in roughly 40% fewer epochs, and setting a new benchmark of 92.4% kNN accuracy from FastSiam’s previously attained value of 90.2.