An Efficient and Compact Network for Simultaneous Multi-Object Tracking and Behavior Monitoring in Pigeon Farming

Animal monitoring plays a crucial role in agriculture, especially in improving animal welfare and farming efficiency. However, existing methods cannot simultaneously achieve video-based tracking for each individual while monitoring their behaviors. To overcome this limitation and meet the demand for automated surveillance in large-scale pigeon farming, this study proposes an efficient and compact network for simultaneous multi-object tracking and behavior monitoring in pigeon farming. The network is capable of tracking each pigeon while recognizing their behaviors in a video sequence. The network combines the DETR detector of the lightweight RepViT backbone with the BoT-SORT motion tracker, which tracks accurately while saving costs. A text-guided multimodal action recognition model is introduced in the action recognition stage, which combines spatiotemporal video features with semantic text embedding to enhance classification accuracy. Experimental results show that the proposed method achieves the optimal tracking performance (IDF1: 96.58%) and action recognition accuracy (Top-1: 94.89%), while reducing the network complexity (13.6M parameters) and computational cost (45.67 GFLOPs). This method provides effective technical support to promote accurate management of poultry farming.