Applying and Optimizing NLP Model with CARU

Ka Hou Chan, Sio Kei Im, Giovanni Pau

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Citations (Scopus)

Abstract

RNN for language models can solve the problem of sparse content and high-dimensional features in traditional N-gram models. However, due to the problems of overfitting and gradient disappearance, the original RNN still lacks long-term content dependence and noise interference. This paper proposes an improved method based on a context word vector for RNN with CARU. In order to alleviate the overfitting problem, a modified DropConnect layer is employed in the proposed model. In addition, the multilayer CARU is used to add contextual word vectors to the model with the feature layer to strengthen the ability to learn long-distance information during the training process. Experimental results show that the proposed method effectively improves the performance of RNN-based language model.

Original languageEnglish
Title of host publication8th International Conference on Advanced Computing and Communication Systems, ICACCS 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1018-1022
Number of pages5
ISBN (Electronic)9781665408165
DOIs
Publication statusPublished - 2022
Event8th International Conference on Advanced Computing and Communication Systems, ICACCS 2022 - Coimbatore, India
Duration: 25 Mar 202226 Mar 2022

Publication series

Name8th International Conference on Advanced Computing and Communication Systems, ICACCS 2022

Conference

Conference8th International Conference on Advanced Computing and Communication Systems, ICACCS 2022
Country/TerritoryIndia
CityCoimbatore
Period25/03/2226/03/22

Keywords

  • DropConnect
  • Language Model
  • Model Analysis
  • Multilayer CARU
  • NLP
  • Word Embedding

Fingerprint

Dive into the research topics of 'Applying and Optimizing NLP Model with CARU'. Together they form a unique fingerprint.

Cite this