Humans display sexual dimorphism across many traits, but little is known about underlying genetic mechanisms and impacts on disease. We utilized single-cell RNA-seq of 480 lymphoblastoid cell lines to reveal that the vast majority (79%) of sex-biased genes are targets of transcription factors that display sex-biased expression. Further, we developed a two-step regression method that identified sex-biased expression quantitative trait loci (sb-eQTL) across the genome. In contrast to previous work, these sb-eQTL are abundant (n=10,754; FDR 5%) and reproducible (replication up to π1=0.56). These sb-eQTL are enriched in over 600 GWAS phenotypes, including 120 sb-eQTL associated with the female-biased autoimmune disease multiple sclerosis. Our results demonstrate widespread genetic impacts on sexual dimorphism and identify possible mechanisms and clinical targets for sex differences in diverse diseases.