This paper explores generative modeling, aiming to simulate diverse examples from an unknown distribution based on observed examples. While recent studies have focused on quantifying the statistical accuracy of popular algorithms, there is a lack of mathematical evaluation regarding the non-replication of observed examples. We present theoretical insights into this aspect, demonstrating that the Wasserstein GAN, constrained to left-invertible push-forward maps, generates distributions that not only avoid replication but also significantly deviate from the empirical distribution. Importantly, we show that left-invertibility achieves this without compromising the statistical optimality of the resulting generator. Our contributions include non-asymptotic results, providing finite sample upper and lower bounds dependent on key parameters such as sample size and dimensionality of the ambient and latent spaces.