We present experimental observations and corresponding numerical simulations illustrating the difference-frequency generation of mid-infrared radiation using few-cycle near-infrared-to-visible pulses, which yields conversion efficiencies above 12% in beta-barium borate crystal. Type I and type II phase-matching are shown to yield qualitatively different intensity-scaling behavior, with the former showing higher overall efficiency, especially with the addition of a zero-order wave plate for modifying the polarization state of the pulse, and the latter having a better stability of the spectrum versus input intensity.