Curculigo orchioides is widely used to treat osteoporosis in China. In this study, we identified the active substances in the crude polysaccharide (CO50) from C. orchioides that had anti-osteoporosis activity in vivo. Two polysaccharides, COP50-1 and COP50-4, were purified from CO50. Based on structural analysis, COP50-1 was composed of α-D-Glcp-(1→, β-D-Galp-(1→, →4)-α-D-Glcp-(1→, →3,4)-α-D-Glcp-(1→, →4,6)-α-D-Glcp-(1→, →4,6)-β-D-Manp-(1→, whereas COP50-4 was composed of α-L-Araf-(1→, →2)-α-L-Rhap-(1→, β-D-Manp-(1→, α-D-Galp-(1→, →2,4)-α-L-Rhap-(1→, →2)-β-D-Manp-(1→, →4)-α-D-GlcAp-(1→, →3)-α-D-GalAp-(1→, →4,6)-α-D-Galp-(1→, →2,3,6)-β-D-Manp-(1→, →2,3,5)-α-L-Araf-(1→, →2,5)-α-L-Araf-(1→, →4)-α-D-Glcp-(1→ and →3)-α-D-Galp-(1→. Pharmacological assessment revealed that COP50-1 had no obvious osteogenic activity. However, COP50-4 (0.5 μM) significantly enhanced the differentiation and mineralization of osteoblasts in vitro. Moreover, the effect of COP50-4 was greater than that of 17β-estradiol. Therefore, COP50-4 may be an effective component of CO50 that has great potential for development as an alternative drug for the treatment of osteoporosis.