Online 3D behavioral tracking of aquatic model organism with a dual-camera system

Zewei Wu, Cui Wang, Wei Zhang, Guodong Sun, Wei Ke, Zhang Xiong

Research output: Contribution to journalArticlepeer-review


Behavioral tracking system of aquatic model organism is crucial for applications in aquaculture, environment and biomedicine, as it facilitates human to monitor subject states by automatically recognizing individual identities, and quantify their movement trajectories. Previous research has been devoted to this topic, but they are still not simple and effective enough. Therefore, this work introduces a novel online monitoring system implemented by dual-camera equipment and software modules consisting of an object detector and a multi-view multi-target tracker. The tracker provides the abilities of cross-view matching, underwater 3D reconstruction, and 3D target tracking. Specifically, our solution adopts a new paradigm, called tracking by early-reconstruction, which prioritizes the 3D reconstruction of targets’ coordinates on a frame-by-frame basis and then tracks them directly in 3D space rather than in a 2D image plane. This paradigm simplifies the complex multi-view tracking problem into a series of local association procedures, allowing us to achieve an online resolution through the iterative approach. To verify the effectiveness of the system, we employ zebrafish as the research subject, and evaluate the accuracy and robustness of the system on tracking benchmark, behavioral tasks and simulated data. Finally, we conducted extensive experiments and demonstrated the efficiency and effectiveness of the proposed system.

Original languageEnglish
Article number102481
JournalAdvanced Engineering Informatics
Publication statusPublished - Aug 2024


  • 3D trajectories reconstruction
  • Deep representation learning
  • Laboratory zebrafish tracking
  • Online monitoring system


Dive into the research topics of 'Online 3D behavioral tracking of aquatic model organism with a dual-camera system'. Together they form a unique fingerprint.

Cite this