Detecting discomfort in infants through facial expressions

Yue Sun, Caifeng Shan, Tao Tan, Tong Tong, Wenjin Wang, Arash Pourtaherian, Peter H.N. De With

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)


Objective: Detecting discomfort status of infants is particularly clinically relevant. Late treatment of discomfort infants can lead to adverse problems such as abnormal brain development, central nervous system damage and changes in responsiveness of the neuroendocrine and immune systems to stress at maturity. In this study, we exploit deep convolutional neural network (CNN) algorithms to address the problem of discomfort detection for infants by analyzing their facial expressions. Approach: A dataset of 55 videos about facial expressions, recorded from 24 infants, is used in our study. Given the limited available data for training, we employ a pre-trained CNN model, which is followed by fine-tuning the networks using a public dataset with labeled facial expressions (the shoulder-pain dataset). The CNNs are further refined with our data of infants. Main results: Using a two-fold cross-validation, we achieve an area under the curve (AUC) value of 0.96, which is substantially higher than the results without any pre-training steps (AUC = 0.77). Our method also achieves better results than the existing method based on handcrafted features. By fusing individual frame results, the AUC is further improved from 0.96 to 0.98. Significance: The proposed system has great potential for continuous discomfort and pain monitoring in clinical practice.

Original languageEnglish
Article number115006
JournalPhysiological Measurement
Issue number11
Publication statusPublished - 3 Dec 2019
Externally publishedYes


  • discomfort detection
  • facial expression recognition
  • transfer learning


Dive into the research topics of 'Detecting discomfort in infants through facial expressions'. Together they form a unique fingerprint.

Cite this