Accurate Facial Landmark Detector via Multi-scale Transformer

Yuyang Sha, Weiyu Meng, Xiaobing Zhai, Can Xie, Kefeng Li

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Facial landmark detection is an essential prerequisite for many face applications, which has attracted much attention and made remarkable progress in recent years. However, some problems still need to be solved urgently, including improving the accuracy of facial landmark detectors in complex scenes, encoding long-range relationships between keypoints and facial components, and optimizing the robustness of methods in unconstrained environments. To address these problems, we propose a novel facial landmark detector via multi-scale transformer (MTLD), which contains three modules: Multi-scale Transformer, Joint Regression, and Structure Loss. The proposed Multi-scale Transformer focuses on capturing long-range information and cross-scale representations from multi-scale feature maps. The Joint Regression takes advantage of both coordinate and heatmap regression, which could boost the inference speed without sacrificing model accuracy. Furthermore, in order to explore the structural dependency between facial landmarks, we design the Structure Loss to fully utilize the geometric information in face images. We evaluate the proposed method through extensive experiments on four benchmark datasets. The results demonstrate that our method outperforms state-of-the-art approaches both in accuracy and efficiency.

Original languageEnglish
Title of host publicationPattern Recognition and Computer Vision - 6th Chinese Conference, PRCV 2023, Proceedings
EditorsQingshan Liu, Hanzi Wang, Rongrong Ji, Zhanyu Ma, Weishi Zheng, Hongbin Zha, Xilin Chen, Liang Wang
PublisherSpringer Science and Business Media Deutschland GmbH
Number of pages13
ISBN (Print)9789819984688
Publication statusPublished - 2024
Event6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023 - Xiamen, China
Duration: 13 Oct 202315 Oct 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14429 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference6th Chinese Conference on Pattern Recognition and Computer Vision, PRCV 2023


  • Facial landmark detection
  • Global information
  • Multi-scale feature
  • Vision transformer


Dive into the research topics of 'Accurate Facial Landmark Detector via Multi-scale Transformer'. Together they form a unique fingerprint.

Cite this