In a previous paper we observed that (classical) tree-level gauge-theory amplitudes can be rearranged to display a duality between color and kinematics. Once this is imposed, gravity amplitudes are obtained using two copies of gauge-theory diagram numerators. Here we conjecture that this duality persists to all quantum loop orders and can thus be used to obtain multiloop gravity amplitudes easily from gauge-theory ones. As a nontrivial test, we show that the three-loop four-point amplitude of N=4 super-Yang-Mills theory can be arranged into a form satisfying the duality, and by taking double copies of the diagram numerators we obtain the corresponding amplitude of N=8 supergravity. We also remark on a nonsupersymmetric two-loop test based on pure Yang-Mills theory resulting in gravity coupled to an antisymmetric tensor and dilaton.