In offline reinforcement learning (RL), the performance of the learned policy highly dependson the quality of offline datasets. However, theoffline dataset contains very limited optimal trajectories in many cases. This poses a challengefor offline RL algorithms, as agents must acquire the ability to transit to high-reward regions.To address this issue, we introduce Diffusion-based Trajectory Stitching (DiffStitch), a noveldiffusion-based data augmentation pipeline thatsystematically generates stitching transitions be-tween trajectories. DiffStitch effectively connects low-reward trajectories with high-rewardtrajectories, forming globally optimal trajectories and thereby mitigating the challenges facedby offline RL algorithms in learning trajectorystitching. Empirical experiments conducted onD4RL datasets demonstrate the effectiveness ofour pipeline across RL methodologies. Notably,DiffStitch demonstrates substantial enhancementsin the performance of one-step methods(IQL),imitation learning methods(TD3+BC) and trajectory optimization methods(DT). Our code ispublicly available at https://anonymous.4open.science/r/DiffStitch-6F22/README.md