IvyGPT: InteractiVe Chinese Pathway Language Model in Medical Domain

Rongsheng Wang, Yaofei Duan, Chan Tong Lam, Jiexin Chen, Jiangsheng Xu, Haoming Chen, Xiaohong Liu, Patrick Cheong Iao Pang, Tao Tan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

General large language models (LLMs) such as ChatGPT have shown remarkable success. However, such LLMs have not been widely adopted for medical purposes, due to poor accuracy and inability to provide medical advice. We propose IvyGPT, an LLM based on LLaMA that is trained and fine-tuned with high-quality medical question-answer (QA) instances and Reinforcement Learning from Human Feedback (RLHF). In the training, we used QLoRA to handle 33 billion parameters on a small number of NVIDIA A100 (80 GB) GPUs. Experimental results show that IvyGPT has outperformed other medical GPT models. The online demo is available at http://81.71.71.157:52022. Our demo video can be found at https://youtu.be/O4D74pQh8Is.

Original languageEnglish
Title of host publicationArtificial Intelligence - Third CAAI International Conference, CICAI 2023, Revised Selected Papers
EditorsLu Fang, Jian Pei, Guangtao Zhai, Ruiping Wang
PublisherSpringer Science and Business Media Deutschland GmbH
Pages378-382
Number of pages5
ISBN (Print)9789819991181
DOIs
Publication statusPublished - 2024
Event3rd CAAI International Conference on Artificial Intelligence, CICAI 2023 - Fuzhou, China
Duration: 22 Jul 202323 Jul 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14474 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference3rd CAAI International Conference on Artificial Intelligence, CICAI 2023
Country/TerritoryChina
CityFuzhou
Period22/07/2323/07/23

Keywords

  • Large language models
  • Medical
  • Reinforcement Learning

Fingerprint

Dive into the research topics of 'IvyGPT: InteractiVe Chinese Pathway Language Model in Medical Domain'. Together they form a unique fingerprint.

Cite this