In recent years, the deep learning (DL) technique has rapidly developed and shown great success in scoring the protein-ligand binding affinities. The protein-ligand conformation optimization based on DL-derived scoring functions holds broad application prospects, for instance, drug design and enzyme engineering. In this study, we evaluated the robustness of a DL-based ligand conformation optimization protocol (DeepRMSD+Vina) for optimizing structures with input perturbations by examining the predicted ligand binding poses and scoring. Our results clearly indicated that compared to traditional optimization algorithms (such as Prime MM-GBSA and Vina optimization), DeepRMSD+Vina exhibits higher performance when treating diverse protein-ligand cases. The DeepRMSD+Vina is robust and can always generate the correct binding structures even if perturbations (up to 3 Å) are introduced to the input structure. The success rate is 62% for perturbation with a RMSD within 2-3 Å. However, the success rate dramatically drops to 11% for large perturbations, with RMSD extending to 3-4 Å. Furthermore, compared to the widely used optimization protocol of AutoDock Vina, the DL-generated conformation shows a balanced performance for all of the systems under examination. Overall, the DL-based DeepRMSD+Vina is unarguably more reliable than the traditional methods, which is attributed to the physically inspired design of the neural networks in DeepRMSD+Vina where the distance-transformed features describing the atomic interactions between the protein and the ligand have been explicitly considered and modeled. The outstanding robustness of the DL-based ligand conformational optimization algorithm further validates its superiority in the field of conformational optimization.