We study an inflationary scenario with a vector field coupled with an inflaton field and show that the inflationary Universe is endowed with anisotropy for a wide range of coupling functions. This anisotropic inflation is a tracking solution where the energy density of the vector field follows that of the inflaton field irrespective of initial conditions. We find a universal relation between the anisotropy and a slow-roll parameter of inflation. Our finding has observational implications and gives a counterexample to the cosmic no-hair conjecture.