Applying and Optimizing NLP Model with CARU

RNN for language models can solve the problem of sparse content and high-dimensional features in traditional N-gram models. However, due to the problems of overfitting and gradient disappearance, the original RNN still lacks long-term content dependence and noise interference. This paper proposes an improved method based on a context word vector for RNN with CARU. In order to alleviate the overfitting problem, a modified DropConnect layer is employed in the proposed model. In addition, the multilayer CARU is used to add contextual word vectors to the model with the feature layer to strengthen the ability to learn long-distance information during the training process. Experimental results show that the proposed method effectively improves the performance of RNN-based language model.