Value Iteration Networks (VINs) enable end-to-end learning for planning tasks, employing a differentiable "planning module" that approximates the value iteration algorithm. Long-term planning, however, remains challenging because very deep VINs are hard to train. To address this problem, we embed Highway Value Iteration—a recent algorithm designed to facilitate long-term credit assignment—into the VINs' structure. This improvement augments the VINs' "planning module" with three additional components: 1) an "aggregate gate," which constructs skip connections to improve information flow across many layers; 2) an "exploration module," crafted to increase the diversity of information and gradient flow in spatial dimensions; 3) a "filter gate" designed to ensure safe exploration. The resulting novel Highway VINs can be trained effectively with hundreds of layers using standard backpropagation. On long-term planning tasks requiring hundreds of planning steps, deep Highway VINs are shown to outperform both traditional VINs and several advanced, very deep neural networks.