Eye tracking has been used to investigate gaze behaviours in individuals with autism spectrum disorder (ASD). However, traditional analysis has yet to find behavioural characteristics shared by both children and adults with ASD. To distinguish core ASD gaze behaviours from those that change with development, we examined temporo-spatial gaze patterns in children and adults with and without ASD while they viewed video clips. We summarized the gaze patterns of 104 participants using multidimensional scaling so that participants with similar gaze patterns would cluster together in a two-dimensional plane. Control participants clustered in the centre, reflecting a standard gaze behaviour, whereas participants with ASD were distributed around the periphery. Moreover, children and adults were separated on the plane, thereby showing a clear effect of development on gaze behaviours. Post hoc frame-by-frame analyses revealed the following findings: (i) both ASD groups shifted their gaze away from a speaker earlier than the control groups; (ii) both ASD groups showed a particular preference for letters; and (iii) typical infants preferred to watch the mouth rather than the eyes during speech, a preference that reversed with development. These results highlight the importance of taking the effect of development into account when addressing gaze behaviours characteristic of ASD.