Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19

Tao Tan; Bipul Das; Ravi Soni; Mate Fejes; Sohan Ranjan; Daniel Attila Szabo; Vikram Melapudi; K. S. Shriram; Utkarsh Agrawal; Laszlo Rusko; Zita Herczeg; Barbara Darazs; Pal Tegzes; Lehel Ferenczi; Rakesh Mullick; Gopal Avinash

doi:10.1007/978-3-030-87234-2_31

Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19

Tao Tan, Bipul Das, Ravi Soni, Mate Fejes, Sohan Ranjan, Daniel Attila Szabo, Vikram Melapudi, K. S. Shriram, Utkarsh Agrawal, Laszlo Rusko, Zita Herczeg, Barbara Darazs, Pal Tegzes, Lehel Ferenczi, Rakesh Mullick, Gopal Avinash

GE Healthcare

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

The COVID-19 pandemic continues to spread and impact the well-being of the global population. The front-line imaging modalities computed tomography (CT) and X-ray play an important role for triaging COVID-19 patients. Considering the limited access to resources (both hardware and trained personnel) and decontamination, CT may not be ideal for triaging suspected subjects. Artificial intelligence (AI) assisted X-ray based applications for triaging and monitoring COVID-19 patients in a timely manner with the additional ability to delineate the disease region boundary are seen as a promising solution. Our proposed solution differs from existing solutions by industry and academic communities. We demonstrates a functional AI model to triage by inferencing using a single x-ray image, while the AI model is trained using both X-ray and CT data. We report on how such a multi-modal training improves the solution compared to X-ray only training. The multi-modal solution increases the AUC (area under the receiver operating characteristic curve) from 0.89 to 0.93 for the classification between COVID-19 and non-COVID-19 cases. It also positively impacts the Dice coefficient (0.59 to 0.62) for segmenting the COVID-19 pathology.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings
Editors	Marleen de Bruijne, Philippe C. Cattin, Stéphane Cotin, Nicolas Padoy, Stefanie Speidel, Yefeng Zheng, Caroline Essert
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	325-334
Number of pages	10
ISBN (Print)	9783030872335
DOIs	https://doi.org/10.1007/978-3-030-87234-2_31
Publication status	Published - 2021
Externally published	Yes
Event	24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 - Virtual, Online Duration: 27 Sept 2021 → 1 Oct 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12907 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021
City	Virtual, Online
Period	27/09/21 → 1/10/21

Keywords

Artificial intelligence
COVID-19
Multi-modal

Access to Document

10.1007/978-3-030-87234-2_31

Cite this

Tan, T., Das, B., Soni, R., Fejes, M., Ranjan, S., Szabo, D. A., Melapudi, V., Shriram, K. S., Agrawal, U., Rusko, L., Herczeg, Z., Darazs, B., Tegzes, P., Ferenczi, L., Mullick, R., & Avinash, G. (2021). Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19. In M. de Bruijne, P. C. Cattin, S. Cotin, N. Padoy, S. Speidel, Y. Zheng, & C. Essert (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings (pp. 325-334). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12907 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-87234-2_31

Tan, Tao ; Das, Bipul ; Soni, Ravi et al. / Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. editor / Marleen de Bruijne ; Philippe C. Cattin ; Stéphane Cotin ; Nicolas Padoy ; Stefanie Speidel ; Yefeng Zheng ; Caroline Essert. Springer Science and Business Media Deutschland GmbH, 2021. pp. 325-334 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a1c2ae847f6c4fadb422d0dcaea25080,

title = "Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19",

abstract = "The COVID-19 pandemic continues to spread and impact the well-being of the global population. The front-line imaging modalities computed tomography (CT) and X-ray play an important role for triaging COVID-19 patients. Considering the limited access to resources (both hardware and trained personnel) and decontamination, CT may not be ideal for triaging suspected subjects. Artificial intelligence (AI) assisted X-ray based applications for triaging and monitoring COVID-19 patients in a timely manner with the additional ability to delineate the disease region boundary are seen as a promising solution. Our proposed solution differs from existing solutions by industry and academic communities. We demonstrates a functional AI model to triage by inferencing using a single x-ray image, while the AI model is trained using both X-ray and CT data. We report on how such a multi-modal training improves the solution compared to X-ray only training. The multi-modal solution increases the AUC (area under the receiver operating characteristic curve) from 0.89 to 0.93 for the classification between COVID-19 and non-COVID-19 cases. It also positively impacts the Dice coefficient (0.59 to 0.62) for segmenting the COVID-19 pathology.",

keywords = "Artificial intelligence, COVID-19, Multi-modal",

author = "Tao Tan and Bipul Das and Ravi Soni and Mate Fejes and Sohan Ranjan and Szabo, {Daniel Attila} and Vikram Melapudi and Shriram, {K. S.} and Utkarsh Agrawal and Laszlo Rusko and Zita Herczeg and Barbara Darazs and Pal Tegzes and Lehel Ferenczi and Rakesh Mullick and Gopal Avinash",

note = "Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 ; Conference date: 27-09-2021 Through 01-10-2021",

year = "2021",

doi = "10.1007/978-3-030-87234-2_31",

language = "English",

isbn = "9783030872335",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "325--334",

editor = "{de Bruijne}, Marleen and Cattin, {Philippe C.} and St{\'e}phane Cotin and Nicolas Padoy and Stefanie Speidel and Yefeng Zheng and Caroline Essert",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings",

address = "Germany",

}

Tan, T, Das, B, Soni, R, Fejes, M, Ranjan, S, Szabo, DA, Melapudi, V, Shriram, KS, Agrawal, U, Rusko, L, Herczeg, Z, Darazs, B, Tegzes, P, Ferenczi, L, Mullick, R & Avinash, G 2021, Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19. in M de Bruijne, PC Cattin, S Cotin, N Padoy, S Speidel, Y Zheng & C Essert (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12907 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 325-334, 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021, Virtual, Online, 27/09/21. https://doi.org/10.1007/978-3-030-87234-2_31

Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19. / Tan, Tao; Das, Bipul; Soni, Ravi et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. ed. / Marleen de Bruijne; Philippe C. Cattin; Stéphane Cotin; Nicolas Padoy; Stefanie Speidel; Yefeng Zheng; Caroline Essert. Springer Science and Business Media Deutschland GmbH, 2021. p. 325-334 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12907 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19

AU - Tan, Tao

AU - Das, Bipul

AU - Soni, Ravi

AU - Fejes, Mate

AU - Ranjan, Sohan

AU - Szabo, Daniel Attila

AU - Melapudi, Vikram

AU - Shriram, K. S.

AU - Agrawal, Utkarsh

AU - Rusko, Laszlo

AU - Herczeg, Zita

AU - Darazs, Barbara

AU - Tegzes, Pal

AU - Ferenczi, Lehel

AU - Mullick, Rakesh

AU - Avinash, Gopal

PY - 2021

Y1 - 2021

N2 - The COVID-19 pandemic continues to spread and impact the well-being of the global population. The front-line imaging modalities computed tomography (CT) and X-ray play an important role for triaging COVID-19 patients. Considering the limited access to resources (both hardware and trained personnel) and decontamination, CT may not be ideal for triaging suspected subjects. Artificial intelligence (AI) assisted X-ray based applications for triaging and monitoring COVID-19 patients in a timely manner with the additional ability to delineate the disease region boundary are seen as a promising solution. Our proposed solution differs from existing solutions by industry and academic communities. We demonstrates a functional AI model to triage by inferencing using a single x-ray image, while the AI model is trained using both X-ray and CT data. We report on how such a multi-modal training improves the solution compared to X-ray only training. The multi-modal solution increases the AUC (area under the receiver operating characteristic curve) from 0.89 to 0.93 for the classification between COVID-19 and non-COVID-19 cases. It also positively impacts the Dice coefficient (0.59 to 0.62) for segmenting the COVID-19 pathology.

AB - The COVID-19 pandemic continues to spread and impact the well-being of the global population. The front-line imaging modalities computed tomography (CT) and X-ray play an important role for triaging COVID-19 patients. Considering the limited access to resources (both hardware and trained personnel) and decontamination, CT may not be ideal for triaging suspected subjects. Artificial intelligence (AI) assisted X-ray based applications for triaging and monitoring COVID-19 patients in a timely manner with the additional ability to delineate the disease region boundary are seen as a promising solution. Our proposed solution differs from existing solutions by industry and academic communities. We demonstrates a functional AI model to triage by inferencing using a single x-ray image, while the AI model is trained using both X-ray and CT data. We report on how such a multi-modal training improves the solution compared to X-ray only training. The multi-modal solution increases the AUC (area under the receiver operating characteristic curve) from 0.89 to 0.93 for the classification between COVID-19 and non-COVID-19 cases. It also positively impacts the Dice coefficient (0.59 to 0.62) for segmenting the COVID-19 pathology.

KW - Artificial intelligence

KW - COVID-19

KW - Multi-modal

UR - http://www.scopus.com/inward/record.url?scp=85116503737&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-87234-2_31

DO - 10.1007/978-3-030-87234-2_31

M3 - Conference contribution

AN - SCOPUS:85116503737

SN - 9783030872335

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 325

EP - 334

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings

A2 - de Bruijne, Marleen

A2 - Cattin, Philippe C.

A2 - Cotin, Stéphane

A2 - Padoy, Nicolas

A2 - Speidel, Stefanie

A2 - Zheng, Yefeng

A2 - Essert, Caroline

PB - Springer Science and Business Media Deutschland GmbH

T2 - 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021

Y2 - 27 September 2021 through 1 October 2021

ER -

Tan T, Das B, Soni R, Fejes M, Ranjan S, Szabo DA et al. Pristine Annotations-Based Multi-modal Trained Artificial Intelligence Solution to Triage Chest X-Ray for COVID-19. In de Bruijne M, Cattin PC, Cotin S, Padoy N, Speidel S, Zheng Y, Essert C, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 325-334. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-87234-2_31