In this paper, we explore the use of fiber bundles extracted from diffusion MR images for a nonlinear registration algorithm. We employ a white matter atlas to automatically label major fiber bundles and to establish correspondence between subjects. We propose a polyaffine framework to calculate a smooth and invertible nonlinear warp field based on these correspondences, and derive an analytical solution for the reorientation of the tensor fields under the polyaffine transformation. We demonstrate our algorithm on a group of subjects and show that it performs comparable to a higher dimensional nonrigid registration algorithm.