Deep-learning based colorectal cancer pathological analysis with hyperspectral light field microscopy

Hao Sheng, Yingying Zhang, Ruixuan Cong, Shuai Wang, Da Yang, Zhenglong Cui, Xuefei Huang, Rongshan Chen, Jiapeng Liu, Wei Ke, Weifeng Lv

Research output: Contribution to journalArticlepeer-review

Abstract

5D hyperspectral light field (H-LF) integrates multi-angular and multi-spectral observation, offering a comprehensive opportunity to capture more detailed information from biological samples. In this article, we integrate hyperspectral light field microscopy imaging to analyze H&E-stained whole slide images (WSIs) of colorectal cancer (CRC). Specifically, we design a triple separable transformer encoder (HLFTST) that efficiently extracts features by decoupling the 5D H-LF data into lower-dimensional components and applying self-attention for global interaction. We also introduce a text encoder-decoder to align H-LF features with language, enabling automatic cell classification and pathology report generation through a three-stage training pipeline. Experiments show our method outperforms 2D, 3D, and 4D baselines, improving precision by up to 4.88% and F1 score by 4.21% across five CRC cell categories. Additionally, it generates meaningful pathology descriptions, highlighting its potential for enhancing diagnostics and supporting personalized treatment in broader biomedical settings.

Original languageEnglish
Article number112987
JournaliScience
Volume28
Issue number8
DOIs
Publication statusPublished - 15 Aug 2025

Keywords

  • Cancer
  • Data engineering
  • Optical imaging

Fingerprint

Dive into the research topics of 'Deep-learning based colorectal cancer pathological analysis with hyperspectral light field microscopy'. Together they form a unique fingerprint.

Cite this