Evidence concerning the association between ambient gaseous air pollutant exposures and semen quality is sparse, and findings in previous studies remain largely inconsistent. We enrolled 1759 men and performed 2184 semen examinations at a large reproductive medical center in Wuhan, China, between 2013 and 2015. Inverse distance weighting interpolation was performed to estimate individual exposures to SO2, NO2, CO, and O3 during the entire period (lag 0-90 days) and key periods (lag 0-9, 10-14, 70-90 days) of sperm development. Linear mixed models were used to analyze exposure-response relationships. SO2 exposure with 0-90 days lag was significantly associated with monotonically decreased sperm concentration (β for each interquartile range increase of exposure: -0.14; 95% CI: -0.23, -0.05), sperm count (-0.21; -0.30, -0.12) and total motile sperm count (-0.16; -0.25, -0.08). Significant associations were observed for total and progressive motility only when SO2 exposure was at the highest quintile (all Ptrend < 0.05). Similar trends were observed for SO2 exposure with 70-90 days lag. NO2, CO, or O3 exposure was not significantly associated with semen quality. Our results suggest that ambient SO2 exposure adversely affects semen quality and highlight the potential to improve semen quality by reducing ambient SO2 exposure during early stages of sperm development.