Models susceptible to spurious correlations in their training data often produce brittle predictions and introduce unintended biases. Addressing this challenge typically involves methods relying on prior knowledge and group annotation to remove spurious correlations, which may not be readily available in many applications. In this paper, we establish a novel connection between unsupervised object-centric learning and mitigation of spurious correlations. Instead of inferring subgroups with varying correlations with the labels, our approach focuses on discovering concepts: discrete ideas that are shared across input samples. Leveraging existing object-centric representation learning, we propose a method that effectively mitigates spurious correlations without requiring human labeling of subgroups. Evaluation on diverse benchmark datasets for subpopulation shifts, without relying on ground-truth or human-annotated groups, demonstrates improvements of 1–2% on the challenging ImageNet-9 background challenge and overall competitive performance in the absence of human-annotated groups.