Occlusion-Robust Multi-Target Tracking and Segmentation Framework with Mask Enhancement

Multi-object tracking stands as one of the most prominent domains in Computer Vision and has significant research value and practical importance. However, due to the complexity of scenarios in the real world, especially in crowded environments with frequent target occlusion, existing MOT frameworks often struggle to achieve precise tracking results. To enhance the trajectory association accuracy of MOT frameworks in occluded scenarios, this paper proposes a mask-enhanced occlusion-robust multi-target tracking and segmentation framework. Our method first introduces a mask-conditional feature fusion network and an occlusion-aware mask propagation network. The former network integrates a mask-guided attention mechanism with a spatial–temporal feature aggregation sub-network to improve tracking robustness in crowded scenes, and the latter network prevents the contamination of online tracking templates from noise inputs by perceiving a target occlusion state. The framework merges the mask-based methods above into a mask-integrated multi-hypothesis tracking algorithm, achieves superior adaptability in occluded scenarios, and enhances the robustness of MOTS tasks. Our framework achieves the best performance on the MOTSA (84.4%), MT, and FN metrics, with a 6.1% reduction in FN compared to the state-of-the-art method. Our method achieves significant improvements in both accuracy and precision and is validated on public datasets.