Long-Term Tracking with Deep Tracklet Association

Yang Zhang, Hao Sheng, Yubin Wu, Shuai Wang, Weifeng Lyu, Wei Ke, Zhang Xiong

Research output: Contribution to journalArticlepeer-review

71 Citations (Scopus)


Recently, most multiple object tracking (MOT) algorithms adopt the idea of tracking-by-detection. Relevant research shows that the performance of the detector obviously affects the tracker, while the improvement of detector is gradually slowing down in recent years. Therefore, trackers using tracklet (short trajectory) are proposed to generate more complete trajectories. Although there are various tracklet generation algorithms, the fragmentation problem still often occurs in crowded scenes. In this paper, we introduce an iterative clustering method that generates more tracklets while maintaining high confidence. Our method shows robust performance on avoiding internal identity switch. Then we propose a deep association method for tracklet association. In terms of motion and appearance, we construct motion evaluation network (MEN) and appearance evaluation network (AEN) to learn long-term features of tracklets for association. In order to explore more robust features of tracklets, a tracklet-based training mechanism is also introduced. Tracklet groups are used as the input of the networks instead of discrete detections. Experimental results show that our training method enhances the performance of the networks. In addition, our tracking framework generates more complete trajectories while maintaining the unique identity of each target as the same time. On the latest MOT 2017 benchmark, we achieve state-of-the-art results.

Original languageEnglish
Article number9096592
Pages (from-to)6694-6706
Number of pages13
JournalIEEE Transactions on Image Processing
Publication statusPublished - 2020


  • Multi-object tracking (MOT)
  • deep association
  • multiple hypothesis tracking (MHT)
  • tracking-by-tracklet


Dive into the research topics of 'Long-Term Tracking with Deep Tracklet Association'. Together they form a unique fingerprint.

Cite this