An Investigation of Multilayer RNNs in Sentiment Analysis

Recurrent Neural Network (RNN) is one of the most powerful deep learning architectures and is commonly used to process various sequential input features, such as video sequences and natural sentence. It outperforms in solving some tasks of Neural Language Processing (NLP) about the sentiment analyse. RNN models have the advantage of allowing to receive data recurrently and extract the main information from the feature encoding of the previous time steps. In this work, there are four types of RNN's units have been analysed, including the Linear RNN, Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU) and Content-Adaptive Recurrent Unit (CARU). The implementation of all these units on multilayer RNN architectures is investigated, and their performance is tested on two benchmark sentiment analysis datasets: IMDB and SST2. The complete source code and experimental results are also provided for future study.