In four experiments, the effect of the semantic relationship between test and inducing stimuli on the magnitude of size contrast in an Ebbinghaus-type illusion was explored. In Experiments 1 and 2, the greatest illusion was found when test and inducing stimuli were identical in shape and differed only in size. Decreased size contrast was found when inducing stimuli were drawn from the same category as the test stimulus, but were not visually identical. Even less size contrast was found when inducing stimuli were from a near conceptual category, with the least effect when they were drawn from a completely different category. In Experiment 3, it was demonstrated that even if test and inducing stimuli are drawn with identical geometric elements, the size contrast illusion is greatly reduced if they represent apparently different conceptual categories (through the manipulation of orientation and perceptual set). In Experiment 4, any geometric or spatial confounds were ruled out. These results suggest that size contrast is strongly influenced by the conceptual similarity between test and inducing stimuli.