In this paper, we provide an idea about how to utilize the deep neural network with large scale social network data to judge the quality of fashion images. Specifically, our aim is to build a deep neural network based model which is able to predict the popularity of fashion-related images. Convolutional Neural Network (CNN) and Multi-layer Perceptron (MLP) are the two major tools to construct the model architecture, in which the CNN is responsible for analyzing images and the MLP is responsible for analyzing other types of social network meta data. Based on this general idea, various tentative model structures are proposed, implemented, and compared in this research. To perform experiments, we constructed a fashion-related dataset which contains over 1 million records from the online social network. Though no real word prediction task has been tried yet, according to the result of dataset-based tests, our models demonstrate good abilities on predicting the popularity of fashion from the online social network using the Xception CNN. However, we also find a very interesting phenomenon, which intuitively indicates there may be limited correlation between popularity and visual design of a fashion due to the power and influence of the online social network.