How to improve the uncertainty estimates in deep models?

Abstract: In spite of the dominant performances of deep neural networks, recent works have shown that they are poorly calibrated, resulting in over-confident predictions. Miscalibration can be exacerbated by overfitting due to the minimization of the cross-entropy during training, as it promotes the predicted softmax probabilities to match the one-hot label assignments. This yields a pre-softmax activation of the correct class that is significantly larger than the remaining activations, potentially leading to overconfident predictions, even when the predicted class is incorrect. This talks aims at presenting popular methods that have been proposed to tackle the miscalibration issue, notably during training time, and understand their impact on the uncertainty estimates provided by the deep models. In particular, we will delve into the use of regularizers, for example in the form of penalty terms, to better model the uncertainty of the network predictions in two fundamental tasks in computer vision: classification and segmentation. Furthermore, I will present how class-wise constraints can be integrated in these problems, presenting simple yet efficient solutions from an optimization standpoint. Last, inspired by the raising popularity of large scale pre-trained vision-language models, we will investigate the impact of several adaptation strategies, such as parameter-efficient adapters, prompt learning and test-time prompt tuning, on the calibration performance compared to zeros-shot CLIP predictions.

Bio: Jose Dolz is an Associate Professor in the Department of Software and IT Engineering at the ETS Montreal, in Canada. Prior to be appointed Professor, Jose was a post-doctoral fellow at the same institution, under the supervision of Prof. Ismail Ben Ayed and Christian Desrosiers. Jose obtained his B.Sc and M.Sc in the Polytechnic University of Valencia, Spain, and his Ph.D. at the University of Lille 2, France, in 2016, where he was recipient of a Marie-Curie Fellowship (2013-2016) to pursue his doctoral studies. His current research focuses on the intersection of deep learning, medical imaging, and computer vision, with a main interest on the optimization, learning strategies with limited supervision and network calibration. Jose has (co-)authored over 90 fully peer-reviewed papers, many of which published in the top venues in medical imaging (MICCAI, IPMI, MedIA, TMI), computer vision (CVPR, ICCV, ECCV) and machine learning (ICML, NeurIPS). Furthermore, Jose has given 5 tutorials on learning with limited supervision at MICCAI (2019-2022) and ICPR(2022), participated in the organization of four summer schools in Deep Learning for Medical Imaging, recognized several times as Outstanding Reviewer (MICCAI'20, ECCV'20, CVPR'21, CVPR'22, NeurIPS'22, ICCV’23), and regular serves as Area Chair in most of these conferences (MICCAI, ECCV, NeurIPS).

Friday August 9th, 11am.
D507