The Library
Multi-scale correlation module for video-based facial expression recognition in the wild
Tools
Li, Tankun, Chan, Kwok-Leung and Tjahjadi, Tardi (2023) Multi-scale correlation module for video-based facial expression recognition in the wild. Pattern Recognition, 142 . 109691. doi:10.1016/j.patcog.2023.109691 ISSN 0031-3203.
PDF
WRAP-Multi-scale-correlation-module-video-facial-expression-recognition-wild-23.pdf - Accepted Version Embargoed item. Restricted access to Repository staff only until 13 May 2024. Contact author directly, specifying your specific needs. - Requires a PDF viewer. Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0. Download (6Mb) |
Official URL: https://doi.org/10.1016/j.patcog.2023.109691
Abstract
The detection of facial muscle movements (e.g., mouth opening) is crucial for facial expression recognition (FER). However, extracting these facial motion features is challenging for a deep-learning recognition system for the following reasons: (1) without explicit labels of motion for training, there is no guarantee that convolutional neural networks (CNNs) can extract motions effectively; (2) compared to human action recognition (e.g., the object moving from left to right), some facial motions (e.g., raising eyebrows) are more subtle and thus harder to extract; and (3) the use of optical flow to extract motion features is time-consuming when using a commonly-used camera. In this work, we propose a Multi-Scale Correlation Module (MSCM) together with an adaptive fusion. Firstly, large as well as small facial motions are extracted by MSCM and encoded by CNNs. Then, an adaptive fusion module is used to aggregate motion features. With these modules, our recognition network is able to model both subtle and large motion features for video-based FER with only the RGB image frames as input. Experiments on two datasets, AFEW and DFEW, show that the network achieves state-of-art performances on the benchmarks.
Item Type: | Journal Article | ||||||||
---|---|---|---|---|---|---|---|---|---|
Subjects: | B Philosophy. Psychology. Religion > BF Psychology Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software T Technology > TA Engineering (General). Civil engineering (General) |
||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Engineering > Engineering | ||||||||
SWORD Depositor: | Library Publications Router | ||||||||
Library of Congress Subject Headings (LCSH): | Human face recognition (Computer science), Pattern recognition systems, Face perception, Optical pattern recognition, Image analysis, Image processing -- Digital techniques, Human-computer interaction | ||||||||
Journal or Publication Title: | Pattern Recognition | ||||||||
Publisher: | Pergamon | ||||||||
ISSN: | 0031-3203 | ||||||||
Official Date: | October 2023 | ||||||||
Dates: |
|
||||||||
Volume: | 142 | ||||||||
Article Number: | 109691 | ||||||||
DOI: | 10.1016/j.patcog.2023.109691 | ||||||||
Status: | Peer Reviewed | ||||||||
Publication Status: | Published | ||||||||
Access rights to Published version: | Restricted or Subscription Access | ||||||||
Date of first compliant deposit: | 16 May 2023 | ||||||||
RIOXX Funder/Project Grant: |
|
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |