Electron-lattice energy exchanges are investigated in gold and silver nanoparticles with sizes ranging from 30 to 2.2 nm embedded in different environments. Femtosecond pump-probe experiments performed in the low-perturbation regime demonstrate a strong increase of the intrinsic electron-phonon interaction for nanoparticles smaller than 10 nm due to a confinement effect.