Human altruistic cooperativeness, one of the most important components of our highly organized society, is along with a greatly enlarged brain relative to body size a spectacular outlier in the animal world. The "social-brain hypothesis" suggests that human brain expansion reflects an increased necessity for information processing to create social reciprocity and cooperation in our complex society. The present study showed that the young adult females (n = 66) showed greater Cooperativeness as well as larger relative global and regional gray matter volumes (GMVs) than the matched males (n = 89), particularly in the social-brain regions including bilateral posterior inferior frontal and left anterior medial prefrontal cortices. Moreover, in females, higher cooperativeness was tightly coupled with the larger relative total GMV and more specifically with the regional GMV in most of the regions revealing larger in female sex-dimorphism. The global and most of regional correlations between GMV and Cooperativeness were significantly specific to female. These results suggest that sexually dimorphic factors may affect the neurodevelopment of these "social-brain" regions, leading to higher cooperativeness in females. The present findings may also have an implication for the pathophysiology of autism; characterized by severe dysfunction in social reciprocity, abnormalities in social-brain, and disproportionately low probability in females.