Transition-metal ion doping has been demonstrated to be effective for tuning the photoluminescence properties of perovskite quantum dots (QDs). However, it would inevitably introduce defects in the lattice. As the Mn concentration increases, the Mn dopant photoluminescence quantum yield (PLQY) first increases and then decreases. Herein the influence of the dopant and the defect states on the photophysics in Mn-doped CsPbCl3 QDs was studied by time-resolved spectroscopies, whereas the energy levels of the possible defect states were analyzed by density functional theory calculations. We reveal the formation of deep interstitials defects (Cli) by Mn2+ doping. The depopulation of initial QD exciton states is a competition between exciton-dopant energy transfer and defect trapping on an early time scale (<100 ps), which determines the final PLQY of the QDs. The present work establishes a robust material optimization guideline for all of the emerging applications where a high PLQY is essential.