Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty

Luca Mossina (1) , Joseba Dalmau (1) , Léo Andéol (2, 3)

1 IRT Saint Exupéry - Institut de Recherche Technologique
2 IMT - Institut de Mathématiques de Toulouse UMR5219
3 SNCF : Innovation & Recherche

We propose a post-hoc, computationally lightweight method to quantify predictive uncertainty in semantic image segmentation. Our approach uses conformal prediction to generate statistically valid prediction sets that are guaranteed to include the ground-truth segmentation mask at a predefined confidence level. We introduce a novel visualization technique of conformalized predictions based on heatmaps, and provide metrics to assess their empirical validity. We demonstrate the effectiveness of our approach on well-known benchmark datasets and image segmentation prediction models, and conclude with practical insights.

Keywords: Conformal prediction; Uncertainty quantification; Semantic image segmentation

Domains: Machine Learning [cs.LG]; Machine Learning [stat.ML]

Luca Mossina, Joseba Dalmau, Léo Andéol. Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty. 2024. ⟨hal-04565173⟩