We investigated the series of temperature and field-driven transitions in LuFe2O4 by optical and Mössbauer spectroscopies, magnetization, and x-ray scattering in order to understand the interplay between charge, structure, and magnetism in this multiferroic material. We demonstrate that charge fluctuation has an onset well below the charge ordering transition, supporting the "order by fluctuation" mechanism for the development of charge order superstructure. Bragg splitting and large magneto-optical contrast suggest a low-temperature monoclinic distortion that can be driven by both temperature and magnetic field.