The Library
SignGraph : a sign sequence is worth graphs of nodes
Tools
Gan, Shiwen, Yin, Yafeng, Jiang, Zhiwei, Wen, Hongkai, Xie, Lei and Lu, Sanglu (2024) SignGraph : a sign sequence is worth graphs of nodes. In: CVPR 2024, Seattle, WA, 17-21 Jun 2024. Published in: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 (In Press)
PDF
WRAP-SignGraph-sign-sequence-worth-graphs-nodes-24.pdf - Accepted Version Embargoed item. Restricted access to Repository staff only - Requires a PDF viewer. Download (1656Kb) |
Abstract
Despite the recent success of sign language research, the widely adopted CNN-based backbones are mainly migrated from other computer vision tasks, in which the contours and texture of objects are crucial for {identifying} objects. They usually treat sign frames as grids and may fail to capture effective cross-region features. In fact, sign language tasks need to focus on the correlation of different regions in one frame and the interaction of different regions among adjacent frames for {identifying} a sign sequence. In this paper, we propose to represent a sign sequence as graphs and introduce a simple yet effective graph-based sign language processing architecture named SignGraph, to extract cross-region features at the graph level. SignGraph consists of two basic modules: Local Sign Graph ($LSG$) module for learning the correlation of \textbf{intra-frame cross-region} features in one frame and Temporal Sign Graph ($TSG$) module for tracking the interaction of \textbf{inter-frame cross-region} features among adjacent frames. With $LSG$ and $TSG$, we build our model in a multiscale manner to ensure that the representation of nodes can capture cross-region features at different granularities. Extensive experiments on current public sign language datasets demonstrate the superiority of our SignGraph model. Our model achieves very
competitive performances with the SOTA model, while not using any extra cues. Code and models are available at: https://github.com/gswycf/SignGraph
Item Type: | Conference Item (Paper) | ||||||
---|---|---|---|---|---|---|---|
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | ||||||
Journal or Publication Title: | Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 | ||||||
Publisher: | IEEE Computer Society | ||||||
Official Date: | 27 February 2024 | ||||||
Dates: |
|
||||||
Status: | Peer Reviewed | ||||||
Publication Status: | In Press | ||||||
Date of first compliant deposit: | 13 March 2024 | ||||||
Conference Paper Type: | Paper | ||||||
Title of Event: | CVPR 2024 | ||||||
Type of Event: | Conference | ||||||
Location of Event: | Seattle, WA | ||||||
Date(s) of Event: | 17-21 Jun 2024 | ||||||
Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |