Skip to main navigation Skip to search Skip to main content

Multi-Annotation Adaption: A Label-Informed Dynamic Framework for Medical Segmentation and Localization

  • Luyi Han
  • , Lishan Cai
  • , Tao Tan
  • , Tianyu Zhang
  • , Yuan Gao
  • , Xin Wang
  • , Chunyao Lu
  • , Xinglong Liang
  • , Antonio Portaluri
  • , Katja Pinker-Domenig
  • , Yue Sun
  • , Jonas Teuwen
  • , Regina Beets-Tan
  • , Sean Benson
  • , Ritse Mann
  • Macao Polytechnic University
  • Netherlands Cancer Institute
  • Radboud University Nijmegen
  • Maastricht University
  • Columbia University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In medical imaging, particularly for lesion segmentation and localization, leveraging weakly supervised learning can significantly reduce the annotation burden. However, the challenge often lies in finding the right balance between annotation time and accuracy. In this study, we address this issue by introducing a balanced approach that uses a multi-annotation pattern. We propose a plug-and-play, label-informed dynamic framework, referred to as MALFOY, designed to efficiently learn from various types of annotations, such as masks, bounding boxes, points, etc. During the training phase, the model is informed of the types or styles of annotations through label weights, which are then linked to the losses for different annotations. Upon completion of training, the optimal segmentation or detection is determined by searching for the best label weights on training records or validation set, guided by specific objective values. Our experiments span two distinct datasets: a polyp dataset comprising 3,515 2D colonoscopy images and a breast cancer dataset containing 1,315 3D DCE-MRI images. The results demonstrate that our proposed model is capable of (1) maximizing the use of diverse annotations through label weights, (2) effectively integrating knowledge from various sources by identifying the optimal weights, (3) performing robustly across both 2D and 3D images while reducing the reliance on precise mask annotations, and (4) can be easily extended for multi-expert annotations. Our code is publicly available https://github.com/fiy2W/MALFOY.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention - MICCAI 2026 - 29th International Conference, 2026, Proceedings
EditorsGuang Yang, Ehsan Adeli, Marleen de Bruijne, Marleen de Bruijne, Bartlomiej W. Papiez, Stefanie Speidel, Pallavi Tiwari, Guoyan Zheng, Mohammad Yaqub, Qi Dou, Islem Rekik
PublisherSpringer Science and Business Media Deutschland GmbH
Pages369-379
Number of pages11
ISBN (Print)9783032380715
DOIs
Publication statusPublished - 2027
Event29th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2026 - Strasbourg, France
Duration: 27 Sept 2026 → 1 Oct 2026

Publication series

NameLecture Notes in Computer Science
Volume16881 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference29th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2026
Country/TerritoryFrance
CityStrasbourg
Period27/09/26 → 1/10/26

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

  1. SDG 3 - Good Health and Well-being
    SDG 3 Good Health and Well-being

Keywords

  • Dynamic Framework
  • Lesion Segmentation
  • Multi-Annotation Adaptive Learning

Fingerprint

Dive into the research topics of 'Multi-Annotation Adaption: A Label-Informed Dynamic Framework for Medical Segmentation and Localization'. Together they form a unique fingerprint.

Cite this