We present a two-step reconstruction method that can qualitatively and quantitatively improve the reconstruction of tissue refractive index (RI) distribution by phase-contrast diffuse optical tomography (PCDOT). In this two-step method, we first recover the distribution of tissue absorption and scattering coefficients by conventional diffuse optical tomography to obtain the geometrical information of lesions, allowing the incorporation of geometrical information as a priori in the PCDOT reconstruction using a locally refined mesh. The method is validated by a series of phantom experiments and evaluated using in vivo data from 42 human subjects. The results demonstrate clear contrast of RI between the lesion and the surroundings, making the image interpretation straightforward. The sensitivity and specificity from these 42 cases are both 81% when RI is used as an imaging parameter for distinguishing between malignant and benign lesions.