The development of deep learning interatomic potentials has enabled efficient and accurate computations in quantum chemistry and materials science, circumventing computationally expensive ab initio calculations. However, the huge number of learnable parameters in deep learning models and their complex architectures hinder physical interpretability and affect the robustness of the derived potential. In this work, we propose graph-EAM, a lightweight graph neural network (GNN) inspired by the empirical embedded atom method to model the interatomic potential of single-element structures. Four material systems: platinum, niobium, silicon, and amorphous-carbon, for which quantum simulation data sets are publicly available, are examined to demonstrate that graph-EAM can achieve high energy and force prediction accuracy─comparable or better than existing state-of-the-art machine learning models─with much fewer parameters. It is also shown that the explicit inclusion of the angular information via three-body atomic density increases the prediction accuracy. The accuracy and efficiency of potentials obtained from graph-EAM can help accelerate the molecular dynamics simulation.