A Novel Distributed Reinforcement Learning Method for Classical Chinese Poetry Generation

Poetry generation has been a classic natural language generation task recently. But so far the methods for this topic mainly imitate and reproduce the poems on the training data set, which indicates that they either have not much connotation or overfit too much like plagiarism of the existing poems. To solve this problem, unlike previous work, instead of tuning the trade-off between connotation and innovation, we propose a distributed reinforcement learning framework, which consists of two stages of training, to generate creative and meaningful poetry. At the first stage we train a model in parallel on a large poetry corpus at word level to master how poets write poems. At the second stage we train the model with a distributed architecture to learn how connotation is developed in human literary art works at sentence level and force the model to imitate itself when it composes some ‘good poems’ to further improve performance. Experiments on generating classical Chinese poetry demonstrate that the proposed model is able to achieve better performance and the high efficiency of training compared to the state-of-the-art.