For the normal physiological responses of the brain or the pathophysiological changes that accompany disease states to be evaluated, it is necessary to compare data sets between different imaging modalities for individual subjects. Similarly, it is important to compare data between individuals both within and across imaging modalities for individual subjects. In a collaborative project with a number of university groups we have developed a system that allows for the within-subject alignment and registration of three-dimensional data sets obtained from different modalities for the same individual. This analysis takes into account the error induced by image acquisition, registration and alignment with regard to scaling, translation and rotation. A more difficult problem is the between-subject warping of individual brain anatomy to match that of another individual or of an idealized model. If the principles of morphometrics and homologous landmarks are applied, three-dimensional brain warping can provide this type of between-subject comparison. The results of accomplishing these two tasks is a system that allows data obtained in a given individual to be compared across structure and function, as obtained from magnetic resonance imaging (MRI) and from positron emission tomography (PET), respectively. It also allows comparison of the resultant information with averaged between-subject data from populations of normal individuals or patients with specific neurological disorders. This system provides the means by which to compare quantitative data between individuals in an objective and automated fashion.