We propose a practical high-dimensional quantum key distribution protocol based on mutually partially unbiased bases utilizing transverse modes of light. In contrast to conventional protocols using mutually unbiased bases, our protocol uses Laguerre-Gaussian and Hermite-Gaussian modes of the same mode order as two mutually partially unbiased bases for encoding, which leads to a scheme free from mode-dependent diffraction in long-distance channels. Since only linear and passive optical elements are needed, our experimental implementation significantly simplifies qudit generation and state measurement. Since this protocol differs from conventional protocols using mutually unbiased bases, we provide a security analysis of our protocol.