Learning to assemble geometric shapes into a larger target structure is a fundamental task with various practical applications. In this work, we tackle this problem by identifying mating surfaces of input point cloud pairs and establishing reliable correspondences between them by leveraging both coarse- and fine-level backbone features. To this end, we introduce Proxy Match Transform (PMT), an approximate high-order feature transform layer that enables reliable correspondences between mating surfaces of shape fragments while incurring low costs in memory and compute. We evaluate the proposed PMTR on the large-scale 3D shape assembly dataset, BreakingBad, and demonstrate its superior performance and efficiency compared to state-of-the-art methods.