Polarization-sensitive optical coherence tomography (PS-OCT) can measure cross-sectional and volumetric images of birefringence in fibrous tissues that provides additional contrast to the intensity images. In this study, we develop polarization-sensitive swept-source OCT (PS-SS-OCT) at 1 microm for deep penetration of the sclera and lamina cribrosa in the posterior part of human eyes. A calibration method for polarization mode dispersion of a circulator, which is employed to conserve the optical power of the interferometer and achieve system sensitivity sufficient for retinal imaging is demonstrated. The A-scan rate, the axial resolution, and the sensitivity of the PS-SS-OCT are 28,000 Hz, 11.0 microm, 94.2 dB, respectively. The posterior part of the eyes of a healthy male subject are measured in vivo. Phase-retardation images show birefringence of deep sclera and lamina cribrosa and enhance the contrast which is not visible in the intensity images. In addition, unlike conventional OCT, our PS-SS-OCT showed polarization-insensitive intensity images, in which an artifact created by the birefringence of sclera has been successfully eliminated.