The control of topological defects in ferroelectrics, in particular by a homogeneous electric field, has emerged as an active research direction. A polarization vortex, which is a fundamental topological defect formed in ferroelectric nanodots, has recently been demonstrated to be switchable by a homogeneous electric field through the control of the built-in electrical distribution using low-symmetry nanodots. Such electrotoroidic switching is investigated for nearly ideal systems, e.g., free-standing nanodots. However, the electrotoroidic switching may be impacted by several factors, for instance, the nanoscale effect of flexoelectricity (intrinsic effect), epitaxial strain and the frequency of the applied field (extrinsic effects). In the present study, the switching of the polarization vortex in a notched nanodot under a homogeneous electric field is investigated. The emphasis is put on a comparison between intrinsic and extrinsic effects on the vortex switching. The results show that the vortex switching takes place through alternate vortex-to-polar and polar-to-vortex transformations due to the appearance of the notch. Although the flexoelectricity breaks the symmetry of the polarization field in the notched nanodot during the polarization transformation and gives rise to an unusual behavior of the vortex core, which departs from the symmetry axis of the notched nanodot, this intrinsic effect plays a relatively insignificant role in the switching behavior of the polarization vortex. In comparison to the intrinsic effect, interestingly, the extrinsic effects strongly influence the vortex switching behavior. Specifically, the frequency of the applied electric field can alter both the shape of the toroidal hysteresis loop and the domain transformation process of the vortex switching. In addition, under substrate constraints, the magnitude of the coercive electric fields at which the vortex-to-polar and polar-to-vortex transformations occur linearly decreases with the increase of strain. The present study provides instructive information on the efficient control of a polarization vortex, which is dominated by extrinsic factors rather than intrinsic ones.