Adaptable Focal Loss for Imbalanced Text Classification

Lu Cao, Xinyue Liu, Hong Shen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)

Abstract

In this paper, we study the problem of imbalanced text classification based on the pre-trained language models. We propose the Adaptable Focal Loss (AFL) method to solve this problem. Firstly, we use the word embeddings from the pre-trained models to construct the sentence level prior by the sum of the word embeddings in the sentence. Then, we extend the Focal Loss, which is widely used in the field of object detection, by replacing the task-special parameters with the scaled-softmax of the distance between the fine-tuned embeddings and the prior embeddings from the pre-trained models. By removing the task-special parameters in Focal Loss, not only can the parameters of arbitrary imbalanced proportion distribution be adjusted automatically according to the task, but also the sentences that are difficult to classify can be given a higher weight. Experimental results show that our methods can easily combine with the common classifier models and significantly improve their performances.

Original languageEnglish
Title of host publicationParallel and Distributed Computing, Applications and Technologies - 22nd International Conference, PDCAT 2021, Proceedings
EditorsHong Shen, Yingpeng Sang, Yong Zhang, Nong Xiao, Hamid R. Arabnia, Geoffrey Fox, Ajay Gupta, Manu Malek
PublisherSpringer Science and Business Media Deutschland GmbH
Pages466-475
Number of pages10
ISBN (Print)9783030967710
DOIs
Publication statusPublished - 2022
Externally publishedYes
Event22nd International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2021 - Guangzhou, China
Duration: 17 Dec 202119 Dec 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13148 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference22nd International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2021
Country/TerritoryChina
CityGuangzhou
Period17/12/2119/12/21

Keywords

  • Adaptive training
  • Focal loss
  • Imbalanced text classification
  • Pre-trained models

Fingerprint

Dive into the research topics of 'Adaptable Focal Loss for Imbalanced Text Classification'. Together they form a unique fingerprint.

Cite this