Exposure to metals, including essential and nonessential elements, is widespread and may be associated with altered semen quality. This study aimed to examine the association between urinary metal concentrations and semen quality in a Chinese population. We measured semen quality parameters (sperm concentration, count, motility, normal morphology, and abnormal head) and 13 metals [arsenic (As), cadmium (Cd), cobalt (Co), chromium (Cr), copper (Cu), iron (Fe), lead (Pb), manganese (Mn), molybdenum (Mo), mercury (Hg), nickel (Ni), selenium (Se), and zinc (Zn)] in the urine of 394 men from an infertility clinic. Multivariable logistic and linear regressions were used to assess the relationship between the creatinine-adjusted urinary metal concentrations and semen quality parameters. We found a significant trend for decreased odds ratios (ORs) for below-reference sperm count with increasing Se quartiles (p for trend = 0.04) and a significant trend for increased sperm percent abnormal head with increasing Ni quartiles (p for trend = 0.03). These associations persisted, even when considering multiple metals. Our results suggest that Ni exposure may be associated with deteriorated sperm morphology and that Se exposure may be associated with better semen quality. However, our findings warrant further studies in a larger and general population.