APDRFormer: Asymmetric Perception Decoupling and Recovery Transformer for multi-modal 3D object detection

In multi-modal 3D object detection, perception asymmetry between camera and LiDAR is observed. Consequently, effective instance cues from the weaker modality are suppressed during BEV fusion. Therefore, modality conflict between classification and regression tasks is induced. Accordingly, an Asymmetric Perception Decoupling and Recovery Transformer is proposed. The novelty lies in jointly recovering fusion-suppressed instance cues and decoupling the modality preferences of classification and regression within a unified framework. Firstly, BEV features are aligned by semantically guided bidirectional optical flow field. Then, local details and global context are jointly modeled by window attention. A reliable basis for subsequent instance recovery is provided. Secondly, a lost instance recovery algorithm is proposed. High-confidence responses suppressed during fusion are mined and recovered. Contextual information of instance is aggregated through deformable attention. Then, the suppressed information is written back into the fused features via cross-attention. Finally, a dual-path task-adaptive decoder is designed. Dedicated feature pathways are constructed separately for classification and regression. Task collaboration is preserved and optimization conflicts are alleviated. Experiments are conducted on the nuScenes and KITTI datasets. On nuScenes test set, 75.4% NDS and 73.2% mAP are achieved. Moreover, robustness in complex traffic scenarios and deployment feasibility are validated on a real-vehicle platform. Code will be available at https://anonymous.4open.science/r/APDRFormer-1048/.