The role of sensory-motor systems in conceptual understanding has been controversial. It has been proposed that many abstract concepts are understood metaphorically through concrete sensory-motor domains such as actions. Using fMRI, we compared neural responses with literal action (Lit; The daughter grasped the flowers), metaphoric action (Met; The public grasped the idea), and abstract (Abs; The public understood the idea) sentences of varying familiarity. Both Lit and Met sentences activated the left anterior inferior parietal lobule, an area involved in action planning, with Met sentences also activating a homologous area in the right hemisphere, relative to Abs sentences. Both Met and Abs sentences activated the left superior temporal regions associated with abstract language. Importantly, activation in primary motor and biological motion perception regions was inversely correlated with Lit and Met familiarity. These results support the view that the understanding of metaphoric action retains a link to sensory-motor systems involved in action performance. However, the involvement of sensory-motor systems in metaphor understanding changes through a gradual abstraction process whereby relatively detailed simulations are used for understanding unfamiliar metaphors, and these simulations become less detailed and involve only secondary motor regions as familiarity increases. Consistent with these data, we propose that anterior inferior parietal lobule serves as an interface between sensory-motor and conceptual systems and plays an important role in both domains. The similarity of abstract and metaphoric sentences in the activation of left superior temporal regions suggests that action metaphor understanding is not completely based on sensory-motor simulations but relies also on abstract lexical-semantic codes.