Psychologists in medical settings are frequently tasked with providing comprehensive evaluations of patients with complex medical and psychiatric conditions. In order to achieve these aims, standardized measures of neurocognitive and psychological functioning are often employed to empirically assess a patient's level of functioning across an array of relevant clinical domains. However, less is known about the degree to which cognitive impairment affects a patient's ability to complete these more comprehensive assessments, raising questions about test validity. The current study sought to contribute to this growing body of literature by examining whether neurocognitive functioning is associated with profile validity on the Personality Assessment Inventory (PAI) in both outpatient (N = 321) and inpatient (N = 131) psychiatric settings. In Study 1, results indicate that while multiple cognitive domains are associated with overall profile validity in psychiatric outpatients, attentional impairment specifically was found to be a significant predictor of profile invalidity after accounting for the effects of overall intellectual functioning (accounting for 13% of the variance overall). The magnitude of attentional impairment specifically, and number of impaired cognitive domains more generally, were also found to be meaningfully associated with overall profile validity. Likewise, in Study 2, PAI profile validity was found to be meaningfully associated with gross cognitive impairment on the WMS-IV Brief Cognitive Status Examination (BCSE) in an inpatient psychiatric setting, with almost half of the patients in the most impaired group yielding invalid PAI profiles. The clinical implications of these findings are discussed.