Trust is a crucial human factor in automated supervisory control tasks. To attain appropriate reliance, the operator's trust should be calibrated to reflect the system's capabilities. This study utilized eye-tracking technology to explore novel approaches, given the intrusive, subjective, and sporadic characteristics of existing trust measurement methods. A real-world scenario of alarm state discrimination was simulated and used to collect eye-tracking data, real-time interaction data, system log data, and subjective trust scale values. In the data processing phase, a dynamic prediction model was hypothesized and verified to deduce and complete the absent scale data in the time series. Ultimately, through eye tracking, a discriminative regression model for trust calibration was developed using a two-layer Random Forest approach, showing effective performance. The findings indicate that this method may evaluate the trust calibration state of operators in human-agent collaborative teams within real-world settings, offering a novel approach to measuring trust calibration. Eye-tracking features, including saccade duration, fixation duration, and the saccade-fixation ratio, significantly impact the assessment of trust calibration status.
Keywords: Random Forest; automated supervisory control; eye tracking; trust calibration; trust measurement.