In the energy range from ∼10(12) eV to ∼10(15) eV, the Galactic cosmic ray flux has anisotropies both on large scales, with an amplitude of the order of 0.1%, and on scales between ≃10° and ≃30°, with amplitudes smaller by a factor of a few. With a diffusion coefficient inferred from Galactic cosmic ray chemical abundances, the diffusion approximation predicts a dipolar anisotropy of comparable size, but does not explain the smaller scale anisotropies. We demonstrate here that energy dependent smaller scale anisotropies naturally arise from the local concrete realization of the turbulent magnetic field within the cosmic ray scattering length. We show how such anisotropies could be calculated if the magnetic field structure within a few tens of parsecs from Earth were known.