Scene-Centric Vehicle Trajectory Prediction at Cooperative Intersection Using Decision-Aware Attention Graph Transformer

Roadside sensors offer a fixed, unobstructed vantage point that can overcome line-of-sight limitations in autonomous driving environments by sharing critical perception data with nearby road agents. While this cooperative approach enhances situational awareness, it also introduces significant computational and communication overhead for autonomous vehicles (AVs). To address this challenge, we propose the Heterogeneous Decision-Aware Attention Graph Transformer (HDAAGT)—a non-autoregressive, encoder-only transformer architecture designed for real-time vehicle trajectory prediction. HDAAGT processes detection data from roadside infrastructure to forecast future vehicle trajectories and communicates these predictions to surrounding agents. By offloading intensive computations from AVs and minimizing transmission latency, our approach improves responsiveness and enables more efficient cooperative perception at intersections and other complex driving scenarios. HDAAGT integrates lane positioning, traffic light states, and vehicle kinematics, enabling a decision-aware graph attention mechanism that models agent-agent and agent-environment interactions. By leveraging a fisheye-based detection and tracking pipeline, our approach eliminates the need for multiple cameras and enables HDAAGT to generate reliable trajectory predictions across the full intersection. We validate our model on the Fisheye-MARC and SinD datasets, demonstrating the capability of HDAAGT in predicting vehicle motion in complex urban intersections with a 1.28 m final displacement error. Additionally, we introduce a new 31k-frame fisheye intersection dataset, the largest of its kind in object tracking, to advance research in intersection-based trajectory prediction.