Performance Evaluation of Text Embeddings with Online Consumer Reviews in Retail Sectors

Analyzing online consumer reviews is one of many popular applications of natural language processing in retail sectors. Text embedding models can transform the textual content of reviews into numerical representations for downstream analytic tasks and therefore they play an essential role in review analytics. As review analytics is increasingly used in the industries, more empirical research is needed to investigate how text embeddings perform in understanding the thoughts and attitudes of customers. In this study, we examined four commonly used text embeddings: namely TF-IDF, word2vec, sent2vec and BERT, to evaluate their performance in predicting the ratings and the sentiments of online consumer reviews. Drawn on the results, we highlight the strengths of these text embeddings and their desirable use cases. Our findings reveal that BERT and sent2vec can produce stable results in predicting the ratings of retail reviews in general. Besides, word2vec is more suitable for identifying negative sentiment within reviews. From a practical perspective, it is worth analyzing reviews from different product categories separately to achieve better results.