Winner-takes-all training is a simple learning paradigm, in which the multiple predictions of so-called hypotheses are leveraged to tackle ambiguous tasks. Recently, a connection was established between winner-takes-all training and centroidal Voronoi tessellations, showing that, once trained, the hypotheses should quantize optimally the shape of the conditional distribution to predict. However, probabilistic reliability guarantees for the predictions are missing. In this work, we show how to take advantage of the appealing geometrical properties of the winner-takes-all learners for conditional density estimation, without modifying its original training scheme. We then discuss the competitiveness of our estimator based on novel theoretical and experimental results on both synthetic and audio data.