In this paper, we present an evaluation of seven automatic brain tissue classifiers based on level of agreements. A number of agreement measures are explained, and we show how they can be used to compare different segmentation techniques. We use the Simultaneous Truth and Performance Level Estimation (STAPLE) of Warfield et al. but also introduce a novel evaluation technique based on the Williams' index. The methods are evaluated using these two techniques on a population of forty subjects, each having an SPGR scan and a co-registered T2 weighted scan. We provide an interpretation of the results and show how similar the output of the STAPLE analysis and Williams' index are. When no ground truth is required, we recommend the use of Williams' index as it is easy and fast to compute.