Electrostatic forces between small groups of colloidal particles are measured using blinking optical tweezers. When the electrostatic screening length is longer than the interparticle separation, forces are found to be non-pairwise-additive. Both pair and multiparticle forces are well described by the linearized Poisson-Boltzmann equation with constant potential boundary conditions. These findings may play an important role in understanding the structure and stability of a wide variety of systems, from micron-sized particles in oil to aqueous nanocolloids.