Cigarette smoking is associated with cognitive decline and dementia, but the extent of the association between smoking and structural brain changes remains unclear. Importantly, it is unknown whether smoking-related brain changes are reversible after smoking cessation. We analyzed data on 504 subjects with recall of lifetime smoking data and a structural brain magnetic resonance imaging at age 73 years from which measures of cortical thickness were extracted. Multiple regression analyses were performed controlling for gender and exact age at scanning. To determine dose-response relationships, the association between smoking pack-years and cortical thickness was tested and then repeated, while controlling for a comprehensive list of covariates including, among others, cognitive ability before starting smoking. Further, we tested associations between cortical thickness and number of years since last cigarette, while controlling for lifetime smoking. There was a diffuse dose-dependent negative association between smoking and cortical thickness. Some negative dose-dependent cortical associations persisted after controlling for all covariates. Accounting for total amount of lifetime smoking, the cortex of subjects who stopped smoking seems to have partially recovered for each year without smoking. However, it took ~25 years for complete cortical recovery in affected areas for those at the mean pack-years value in this sample. As the cortex thins with normal aging, our data suggest that smoking is associated with diffuse accelerated cortical thinning, a biomarker of cognitive decline in adults. Although partial recovery appears possible, it can be a long process.