Whether the amygdala responds in a stimulus- or a task-specific way, and how it corresponds to such effects in sensory cortices is dubious. Eighteen volunteers participated in a functional magnetic resonance imaging study in which they were asked to identify either emotion or gender in visually presented scenes, faces, and sentences. Amygdala and the lateral occipital complex showed similar stimulus effect with greater activation to scenes than to faces and sentences, whereas the superior temporal complex responded preferentially to sentences. No task effect was observed in the amygdala, whereas lateral occipital complex and superior temporal complex showed left lateralized selectivity to the emotional task. These results suggest that the amygdala is more sensitive to stimulus than explicit task parameters when processing emotion.