Automated discomfort detection for premature infants in NICU using time-frequency feature-images and CNNs

Yue Sun, Deedee Kommers, Tao Tan, Wenjin Wang, Xi Long, Caifeng Shan, Carola Van Pul, Ronald M. Aarts, Peter Andriessen, Peter H.N. De With

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)


Pain or discomfort exposure during hospitalization of preterm infants has an adverse effect on brain development. Contactless monitoring has been considered to be a promising approach for detecting infant pain and discomfort moments continuously. In this study, our main objective is to develop an automated discomfort detection system based on video monitoring, allowing caregivers to provide timely and appropriate treatments. The system first employs the optical ow to estimate infant body motion trajectories across video frames. Following the movement estimation, Log Mel-spectrogram, Mel Frequency Cepstral Coefficients (MFCCs) and Spectral Subband Centroid Frequency (SSCF) features are calculated from the One-Dimensional (1D) motion signal. These features enable the representation of the 1D motion signals by Two-Dimensional (2D) time-frequency representations of the distribution of signal energy. Finally, deep Convolutional Neural Networks (CNNs) are applied on the 2D images for the binary - comfort/discomfort classification. The performance of the model is assessed using leave-one-infant- out cross-validation. Our algorithm was evaluated on a dataset containing 183 video segments recorded from 11 infants during 17 heel prick events, which is a pain stimulus associated with a routine care procedure. Experimental results showed an area under the receiver operating characteristic curve of 0.985 and an accuracy of 94.2%, which offers a promising possibility to deploy the proposed system in clinical practice.

Original languageEnglish
Title of host publicationMedical Imaging 2020
Subtitle of host publicationComputer-Aided Diagnosis
EditorsHorst K. Hahn, Maciej A. Mazurowski
ISBN (Electronic)9781510633957
Publication statusPublished - 2020
Externally publishedYes
EventMedical Imaging 2020: Computer-Aided Diagnosis - Houston, United States
Duration: 16 Feb 202019 Feb 2020

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2020: Computer-Aided Diagnosis
Country/TerritoryUnited States


  • computer-aided diagnosis
  • convolutional neural networks
  • deep learning
  • discomfort detection
  • infant discomfort


Dive into the research topics of 'Automated discomfort detection for premature infants in NICU using time-frequency feature-images and CNNs'. Together they form a unique fingerprint.

Cite this