Abstract: We propose VecKM, a novel local point cloud encoder that is efficient and robust to noise. VecKM leverages a unique approach by vectorizing a kernel mixture to represent local point clouds. This technique not only enhances the descriptiveness of the encoding but also ensures its robustness to noise. The efficacy of VecKM is supported by two theorems that confirm its ability to reconstruct and preserve the similarity of the shape. Besides, VecKM significantly reduces memory costs compared to traditional multi-layer perceptron-based encoders by a factor of $K$, where $K$ is the neighborhood size. This efficiency is primarily due to VecKM's unique factorizable property. Through extensive evaluations, VecKM demonstrates not only > 40% increase in computation speed but also superior scalability compared to existing encoders. When integrated into deep point cloud architectures, VecKM achieves speeds up to 70x faster than traditional models like PointNet++ or point transformers, without sacrificing accuracy. In the semantic segmentation task, VecKM improves the performance of the PointNet++ baseline, achieving a 6.0% increase in mIoU.