TY - GEN
T1 - Explainable and Controllable Motion Curve Guided Cardiac Ultrasound Video Generation
AU - Yu, Junxuan
AU - Chen, Rusi
AU - Zhou, Yongsong
AU - Chen, Yanlin
AU - Duan, Yaofei
AU - Huang, Yuhao
AU - Zhou, Han
AU - Tan, Tao
AU - Yang, Xin
AU - Ni, Dong
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
PY - 2025
Y1 - 2025
N2 - Echocardiography video is a primary modality for diagnosing heart diseases, but the limited data poses challenges for both clinical teaching and machine learning training. Recently, video generative models have emerged as a promising strategy to alleviate this issue. However, previous methods often relied on holistic conditions during generation, hindering the flexible movement control over specific cardiac structures. In this context, we propose an explainable and controllable method for echocardiography video generation, taking an initial frame and a motion curve as guidance. Our contributions are three-fold. First, we extract motion information from each heart substructure to construct motion curves, enabling the diffusion model to synthesize customized echocardiography videos by modifying these curves. Second, we propose the structure-to-motion alignment module, which can map semantic features onto motion curves across cardiac structures. Third, The position-aware attention mechanism is designed to enhance video consistency utilizing Gaussian masks with structural position information. Extensive experiments on three echocardiography datasets show that our method outperforms others regarding fidelity and consistency. The full code will be released at https://github.com/mlmi-2024-72/ECM.
AB - Echocardiography video is a primary modality for diagnosing heart diseases, but the limited data poses challenges for both clinical teaching and machine learning training. Recently, video generative models have emerged as a promising strategy to alleviate this issue. However, previous methods often relied on holistic conditions during generation, hindering the flexible movement control over specific cardiac structures. In this context, we propose an explainable and controllable method for echocardiography video generation, taking an initial frame and a motion curve as guidance. Our contributions are three-fold. First, we extract motion information from each heart substructure to construct motion curves, enabling the diffusion model to synthesize customized echocardiography videos by modifying these curves. Second, we propose the structure-to-motion alignment module, which can map semantic features onto motion curves across cardiac structures. Third, The position-aware attention mechanism is designed to enhance video consistency utilizing Gaussian masks with structural position information. Extensive experiments on three echocardiography datasets show that our method outperforms others regarding fidelity and consistency. The full code will be released at https://github.com/mlmi-2024-72/ECM.
UR - http://www.scopus.com/inward/record.url?scp=85208442214&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-73290-4_23
DO - 10.1007/978-3-031-73290-4_23
M3 - Conference contribution
AN - SCOPUS:85208442214
SN - 9783031732928
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 232
EP - 241
BT - Machine Learning in Medical Imaging - 15th International Workshop, MLMI 2024, Held in Conjunction with MICCAI 2024, Proceedings
A2 - Xu, Xuanang
A2 - Cui, Zhiming
A2 - Sun, Kaicong
A2 - Rekik, Islem
A2 - Ouyang, Xi
PB - Springer Science and Business Media Deutschland GmbH
T2 - 15th International Workshop on Machine Learning in Medical Imaging, MLMI 2024 was held in conjunction with the 27th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2024
Y2 - 6 October 2024 through 6 October 2024
ER -