In addressing the challenge of Crystal Structure Prediction (CSP), symmetry-aware CSP deep learning models have been extensively studied, particularly diffusion models, which treat CSP as a conditional generation task. However, ensuring permutation, rotation, and periodic translation equivariance during diffusion process remains incompletely addressed. In this work, we propose EDCSP, a novel equivariant diffusion generative model. We address the overlooked issue of lattice permutation equivariance in existing models. Specifically, during diffusion process, when lattice parameters are permuted, the atomic fractional coordinates undergo an equivariant transformation. Additionally, we develop a unique noising algorithm that rigorously maintains periodic translation equivariance throughout both inference and training processes. Our experiments indicate that EDCSP significantly surpasses existing models in terms of generating accurate structures and demonstrates faster convergence during the training process.