Insulin resistance caused by the overexpression of protein tyrosine phosphatase 1 B (PTP1B) as well as the dephosphorylation of its target is one of the main causes of type 2 diabetes (T2D). A newly discovered proteoglycan, Fudan-Yueyang Ganoderma lucidum (FYGL) extracted from Ganoderma lucidum, was first reported to be capable of competitively inhibiting PTP1B activity in vitro in our previous work. In the present study, we sought to reveal the mechanism of PTP1B inhibition by FYGL at the animal and cellular levels. We found that FYGL can decrease blood glucose, reduce body weight and ameliorate insulin resistance in ob/ob mice. Decrease of PTP1B expression and increase of the phosphorylation of PTP1B targets in the insulin signaling pathway of skeletal muscles were observed. In order to clearly reveal the underlying mechanism of the hypoglycemic effect caused by FYGL, we further investigated the effects of FYGL on the PTP1B-involved insulin signaling pathway in rat myoblast L6 cells. We demonstrated that FYGL had excellent cell permeability by using a confocal laser scanning microscope and a flow cytometer. We found that FYGL had a positive effect on insulin-stimulated glucose uptake by using the 2-deoxyglucose (2-DG) method. FYGL could inhibit PTP1B expression at the mRNA level, phosphorylating insulin receptor substrate-1 (IRS1), as well as activating phosphatidylinositol-3 kinase (PI3K) and protein kinase B (Akt). Finally, FYGL increased the phosphorylation of adenosine monophosphate-activated protein kinase (AMPK) and consequently up-regulated the expression of glucose transporter type 4 (GLUT4), promoting GLUT4 transportation to the plasma membrane in PTP1B-transfected L6 cells. Our study provides theoretical evidence for FYGL to be potentially used in T2D management.