Incidence of triple-negative breast cancer (TNBC), which is cancer without expression of ER, PgR, and HER2, and nuclear grade (NG) are closely correlated with malignant potential of breast cancer. However, criteria to determine aggressiveness of breast cancer based on these factors have not been elucidated. The aim of this study was to create criteria using these factors to predict tumor recurrence in invasive ductal carcinoma (IDC) of the breast. One hundred and seventy-nine patients with IDC of the breast, which had been treated by surgical resection, were included. One point was added for each factor of the two categories of TNBC and NG 3. The sum of the scores (TGS 0, 1, or 2) was calculated. Significant difference was observed between TGS and the incidence of tumor recurrence (P < 0.0001). Moreover, significant differences were observed regarding relapse-free survival (RFS) between patients with TGS 0 and TGS 1 (P < 0.0001) and patients with TGS 1 and TGS 2 (P = 0.024). TGS might contain a clinical advantage as a useful predictor for tumor recurrence of IDC of the breast and could classify prognosis of the patients with a preferable stratification.