The paper presents a visual analytics approach for analysing eye-tracking data recorded from multiple viewers watching dynamic stimuli (video). The workflow combines automatic preprocessing with interactive spatiotemporal exploration in a space-time cube (STC) visualisation supported by multiple coordinated views.
Preprocessing. Dense optical flow is computed between consecutive video frames using the Farnebäck method. Shot boundaries are then detected at positions of high disturbance in the flow field. Per-frame spatial density is characterised by computing each gaze point's distance to the centre of mass of all gaze points in that frame. The gaze data are segmented by the detected shot boundaries, and mean-shift clustering is applied independently within each shot, yielding potential areas of interest (AOIs) without requiring a predefined number of clusters. The shot-based separation prevents gaze-reorientation points after cuts from being falsely assigned to clusters of the preceding shot.
Visualisation. The preprocessed data are arranged in a freely rotatable 3D space-time cube: the horizontal plane encodes the video frame's spatial coordinates and the vertical axis encodes time. Data points are colour-mapped by their density value (Gaussian kernel on distance to centre of mass); clusters appear as smoothed axis-aligned box hulls with size labels. Adjustable 2D wall projections provide occlusion-free overviews. A linked video preview shows dynamic AOIs overlaid on the current frame, and shot boundary key-frames support direct temporal navigation. Standard visualisations—static and dynamic heat maps and individual scan path lines—are also integrated.
Interactive exploration. The analyst enters an iterative exploration cycle using three complementary strategies observed in the user study: (1) adjusting the kernel size σ to filter data points by spatial density, revealing moments of high attentional synchrony; (2) filtering clusters by size to locate the most prominent AOIs and navigate to their temporal positions; (3) navigating to shot boundaries to examine gaze reorientation latencies and centre bias. Additionally, the analyst can select a time span and key-frame to generate a motion-compensated heat map—gaze points are traced along the optical-flow field to the key-frame, accumulating attention on the objects actually observed rather than along motion trails. Through these operations the analyst identifies patterns: periods of attentional synchrony, multiple concurrent AOIs with relative attention distribution, object-tracking motion signatures, and gaze reorientation behaviour at cuts. The cycle repeats with different parameter settings and temporal foci until the analyst has characterised the salient viewing behaviours in the stimulus.