Adaptable Focal Loss for Imbalanced Text Classification

Lu Cao, Xinyue Liu, Hong Shen

研究成果: Conference contribution同行評審

2 引文 斯高帕斯(Scopus)

摘要

In this paper, we study the problem of imbalanced text classification based on the pre-trained language models. We propose the Adaptable Focal Loss (AFL) method to solve this problem. Firstly, we use the word embeddings from the pre-trained models to construct the sentence level prior by the sum of the word embeddings in the sentence. Then, we extend the Focal Loss, which is widely used in the field of object detection, by replacing the task-special parameters with the scaled-softmax of the distance between the fine-tuned embeddings and the prior embeddings from the pre-trained models. By removing the task-special parameters in Focal Loss, not only can the parameters of arbitrary imbalanced proportion distribution be adjusted automatically according to the task, but also the sentences that are difficult to classify can be given a higher weight. Experimental results show that our methods can easily combine with the common classifier models and significantly improve their performances.

原文English
主出版物標題Parallel and Distributed Computing, Applications and Technologies - 22nd International Conference, PDCAT 2021, Proceedings
編輯Hong Shen, Yingpeng Sang, Yong Zhang, Nong Xiao, Hamid R. Arabnia, Geoffrey Fox, Ajay Gupta, Manu Malek
發行者Springer Science and Business Media Deutschland GmbH
頁面466-475
頁數10
ISBN(列印)9783030967710
DOIs
出版狀態Published - 2022
對外發佈
事件22nd International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2021 - Guangzhou, China
持續時間: 17 12月 202119 12月 2021

出版系列

名字Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
13148 LNCS
ISSN(列印)0302-9743
ISSN(電子)1611-3349

Conference

Conference22nd International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2021
國家/地區China
城市Guangzhou
期間17/12/2119/12/21

指紋

深入研究「Adaptable Focal Loss for Imbalanced Text Classification」主題。共同形成了獨特的指紋。

引用此