Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis

Yue Sun; Deedee Kommers; Wenjin Wang; Rohan Joshi; Caifeng Shan; Tao Tan; Ronald M. Aarts; Carola Van Pul; Peter Andriessen; Peter H.N. De With

doi:10.1109/EMBC.2019.8857597

Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis

Yue Sun, Deedee Kommers, Wenjin Wang, Rohan Joshi, Caifeng Shan, Tao Tan, Ronald M. Aarts, Carola Van Pul, Peter Andriessen, Peter H.N. De With

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

20 Citations (Scopus)

Abstract

Frequent pain and discomfort in premature infants can lead to long-term adverse neurodevelopmental outcomes. Video-based monitoring is considered to be a promising contactless method for identification of discomfort moments. In this study, we propose a video-based method for automated detection of infant discomfort. The method is based on analyzing facial and body motion. Therefore, motion trajectories are estimated from frame to frame using optical flow. For each video segment, we further calculate the motion acceleration rate and extract 18 time- and frequency-domain features characterizing motion patterns. A support vector machine (SVM) classifier is then applied to video sequences to recognize infant status of comfort or discomfort. The method is evaluated using 183 video segments for 11 infants from 17 heel prick events. Experimental results show an AUC of 0.94 for discomfort detection and the average accuracy of 0.86 when combining all proposed features, which is promising for clinical use.

Original language	English
Title of host publication	2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	5995-5999
Number of pages	5
ISBN (Electronic)	9781538613115
DOIs	https://doi.org/10.1109/EMBC.2019.8857597
Publication status	Published - Jul 2019
Externally published	Yes
Event	41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019 - Berlin, Germany Duration: 23 Jul 2019 → 27 Jul 2019

Publication series

Name	Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
ISSN (Print)	1557-170X

Conference

Conference	41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019
Country/Territory	Germany
City	Berlin
Period	23/07/19 → 27/07/19

Access to Document

10.1109/EMBC.2019.8857597

Cite this

Sun, Y., Kommers, D., Wang, W., Joshi, R., Shan, C., Tan, T., Aarts, R. M., Van Pul, C., Andriessen, P., & De With, P. H. N. (2019). Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019 (pp. 5995-5999). Article 8857597 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/EMBC.2019.8857597

Sun, Yue ; Kommers, Deedee ; Wang, Wenjin et al. / Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 5995-5999 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS).

@inproceedings{d766cdf5d2594043943b757260789812,

title = "Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis",

abstract = "Frequent pain and discomfort in premature infants can lead to long-term adverse neurodevelopmental outcomes. Video-based monitoring is considered to be a promising contactless method for identification of discomfort moments. In this study, we propose a video-based method for automated detection of infant discomfort. The method is based on analyzing facial and body motion. Therefore, motion trajectories are estimated from frame to frame using optical flow. For each video segment, we further calculate the motion acceleration rate and extract 18 time- and frequency-domain features characterizing motion patterns. A support vector machine (SVM) classifier is then applied to video sequences to recognize infant status of comfort or discomfort. The method is evaluated using 183 video segments for 11 infants from 17 heel prick events. Experimental results show an AUC of 0.94 for discomfort detection and the average accuracy of 0.86 when combining all proposed features, which is promising for clinical use.",

author = "Yue Sun and Deedee Kommers and Wenjin Wang and Rohan Joshi and Caifeng Shan and Tao Tan and Aarts, {Ronald M.} and {Van Pul}, Carola and Peter Andriessen and {De With}, {Peter H.N.}",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019 ; Conference date: 23-07-2019 Through 27-07-2019",

year = "2019",

month = jul,

doi = "10.1109/EMBC.2019.8857597",

language = "English",

series = "Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "5995--5999",

booktitle = "2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019",

address = "United States",

}

Sun, Y, Kommers, D, Wang, W, Joshi, R, Shan, C, Tan, T, Aarts, RM, Van Pul, C, Andriessen, P & De With, PHN 2019, Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis. in 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019., 8857597, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, Institute of Electrical and Electronics Engineers Inc., pp. 5995-5999, 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019, Berlin, Germany, 23/07/19. https://doi.org/10.1109/EMBC.2019.8857597

Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis. / Sun, Yue; Kommers, Deedee; Wang, Wenjin et al.
2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 5995-5999 8857597 (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis

AU - Sun, Yue

AU - Kommers, Deedee

AU - Wang, Wenjin

AU - Joshi, Rohan

AU - Shan, Caifeng

AU - Tan, Tao

AU - Aarts, Ronald M.

AU - Van Pul, Carola

AU - Andriessen, Peter

AU - De With, Peter H.N.

PY - 2019/7

Y1 - 2019/7

N2 - Frequent pain and discomfort in premature infants can lead to long-term adverse neurodevelopmental outcomes. Video-based monitoring is considered to be a promising contactless method for identification of discomfort moments. In this study, we propose a video-based method for automated detection of infant discomfort. The method is based on analyzing facial and body motion. Therefore, motion trajectories are estimated from frame to frame using optical flow. For each video segment, we further calculate the motion acceleration rate and extract 18 time- and frequency-domain features characterizing motion patterns. A support vector machine (SVM) classifier is then applied to video sequences to recognize infant status of comfort or discomfort. The method is evaluated using 183 video segments for 11 infants from 17 heel prick events. Experimental results show an AUC of 0.94 for discomfort detection and the average accuracy of 0.86 when combining all proposed features, which is promising for clinical use.

AB - Frequent pain and discomfort in premature infants can lead to long-term adverse neurodevelopmental outcomes. Video-based monitoring is considered to be a promising contactless method for identification of discomfort moments. In this study, we propose a video-based method for automated detection of infant discomfort. The method is based on analyzing facial and body motion. Therefore, motion trajectories are estimated from frame to frame using optical flow. For each video segment, we further calculate the motion acceleration rate and extract 18 time- and frequency-domain features characterizing motion patterns. A support vector machine (SVM) classifier is then applied to video sequences to recognize infant status of comfort or discomfort. The method is evaluated using 183 video segments for 11 infants from 17 heel prick events. Experimental results show an AUC of 0.94 for discomfort detection and the average accuracy of 0.86 when combining all proposed features, which is promising for clinical use.

UR - http://www.scopus.com/inward/record.url?scp=85076714823&partnerID=8YFLogxK

U2 - 10.1109/EMBC.2019.8857597

DO - 10.1109/EMBC.2019.8857597

M3 - Conference contribution

C2 - 31947213

AN - SCOPUS:85076714823

T3 - Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS

SP - 5995

EP - 5999

BT - 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019

Y2 - 23 July 2019 through 27 July 2019

ER -

Sun Y, Kommers D, Wang W, Joshi R, Shan C, Tan T et al. Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 5995-5999. 8857597. (Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS). doi: 10.1109/EMBC.2019.8857597

Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this