Online 3D behavioral tracking of aquatic model organism with a dual-camera system

Behavioral tracking system of aquatic model organism is crucial for applications in aquaculture, environment and biomedicine, as it facilitates human to monitor subject states by automatically recognizing individual identities, and quantify their movement trajectories. Previous research has been devoted to this topic, but they are still not simple and effective enough. Therefore, this work introduces a novel online monitoring system implemented by dual-camera equipment and software modules consisting of an object detector and a multi-view multi-target tracker. The tracker provides the abilities of cross-view matching, underwater 3D reconstruction, and 3D target tracking. Specifically, our solution adopts a new paradigm, called tracking by early-reconstruction, which prioritizes the 3D reconstruction of targets’ coordinates on a frame-by-frame basis and then tracks them directly in 3D space rather than in a 2D image plane. This paradigm simplifies the complex multi-view tracking problem into a series of local association procedures, allowing us to achieve an online resolution through the iterative approach. To verify the effectiveness of the system, we employ zebrafish as the research subject, and evaluate the accuracy and robustness of the system on tracking benchmark, behavioral tasks and simulated data. Finally, we conducted extensive experiments and demonstrated the efficiency and effectiveness of the proposed system.