Self-incompatibility (SI) is a widespread mechanism that prevents inbreeding in flowering plants. In many species, SI is controlled by a single locus (the S locus) where numerous alleles are maintained by negative frequency-dependent selection. Inbreeding depression, the decline in fitness of selfed individuals compared to outcrossed ones, is an essential factor in the evolution of SI systems. Conversely, breeding systems influence levels of inbreeding depression. Little is known about the joint effect of SI and drift on inbreeding depression. Here we studied, using a two-locus model, the effect of SI (frequency-dependent selection) on a locus subject to recurrent deleterious mutations causing inbreeding depression. Simulations were performed to assess the effect of population size and linkage between the two loci on the level of inbreeding depression and genetic load. We show that the sheltering of deleterious alleles linked to the S locus strengthens inbreeding depression in small populations. We discuss the implications of our results for the evolution of SI systems.