BDCNet: Feature-decoupling and cross-task collaboration network with biological priors for cell segmentation and classification

In histopathology, cell segmentation and classification based on H&E-stained images are fundamental clinical tasks. Current methods decompose cell segmentation into three subtasks: nuclear shape extraction, overlapping nuclei separation, and nuclear type recognition, which adopt an encoder-decoder-Segmentation Head (SegHead) architecture, and the outputs of each task are integrated through post-processing to achieve final cell segmentation. However, these methods face several limitations: (1) encoders fail to consider the spatial relationships among cells, which limits their ability to extract features of medical images; (2) decoders are structurally simple and functionally limited, making them incapable of capturing the diverse features required by different tasks; (3) SegHeads are independent of each other, which limits the model’s ability to leverage similarity-based features to optimize the subtask outputs. To address these issues, we propose a feature-decoupling and cross-task collaboration network with biological priors for cell segmentation and classification (BDCNet). First, we expand the dataset by preserving biological priors derived from cell clusters, and employ GNNs to embed these priors in order to provide spatial relationships among nuclei and category similarity information for the encoder. Moreover, to extract distinct feature information, we design a Regional Pixel Difference Convolution (RPDC) algorithm, which partitions characteristic regions to capture distinct information related to shape, gradient, and texture, and further propose a series of feature decoupling modules based on RPDC. Furthermore, to promote feature complementarity across tasks, we introduce Cross-Task Consistency Module (CTCM), enhancing semantic consistency and collaboration among tasks to improve segmentation performance. Extensive experiments show that our method outperforms previous state-of-the-art methods on two public cell segmentation datasets.