This study examined the electrophysiological bases of the effect of language on color perception. In a visual search task, a target was presented to the left or right visual field. The target color was either from the same category as a set of distractors (within-category condition) or from a different category (between-category condition). For both category conditions, the targets elicited a clear N2pc (N2-posterior-contralateral) component in the event-related potential (ERP) in the contralateral hemisphere. In the left hemisphere only, the N2pc amplitude for the between-category condition was larger than that for the within-category condition. These results indicate that the N2pc could be used as an index to describe the lateralization effect of language on color perception.