End-To-End Deep Learning for pNN50 Estimation Using a Spatiotemporal Representation

Sayyedjavad Ziaratnia, Peeraya Sripian, Tipporn Laohakangvalvit, Kazuo Ohzeki, Midori Sugaya

研究成果: Conference contribution

2 被引用数 (Scopus)


Various industries widely use emotion estimation to evaluate their consumer satisfaction towards their products. Generally, emotion can be estimated based on observable expressions such as facial expression, or unobservable expressions such as biological signals. Although used by many research, the Facial Expression Recognition has a lack of precision for expressions that are very similar to each other or a situation where the shown expression differs from the real subject’s emotion. On the other hand, biological signal indexes such as pNN50 can act as a supportive mechanism to improve emotion estimation from observable expressions such as facial expression recognition method. pNN50 is a reliable index to estimate stress-relax, and it originates from unconscious emotions that cannot be manipulated. In this work, we propose a method for pNN50 estimation from facial video using a Deep Learning model. Transfer learning technique and a pre-trained Image recognition Convolutional Neural Network (CNN) model are employed to estimate pNN50 based on a spatiotemporal map created from a series of frames in a facial video. The model which trained on low, middle, and high pNN50 values, shows an accuracy of about 80%. Therefore, it indicates the potential of our proposed method, and we can expand it to categorize the more detailed level of pNN50 values.

ホスト出版物のタイトルHCI International 2021 - Posters - 23rd HCI International Conference, HCII 2021, Proceedings
編集者Constantine Stephanidis, Margherita Antona, Stavroula Ntoa
出版社Springer Science and Business Media Deutschland GmbH
出版ステータスPublished - 2021
イベント23rd International Conference on Human-Computer Interaction, HCII 2021 - Virtual, Online
継続期間: 2021 7月 242021 7月 29


名前Communications in Computer and Information Science


Conference23rd International Conference on Human-Computer Interaction, HCII 2021
CityVirtual, Online

ASJC Scopus subject areas

  • コンピュータ サイエンス(全般)
  • 数学 (全般)


「End-To-End Deep Learning for pNN50 Estimation Using a Spatiotemporal Representation」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。