Effective Adapter for Face Recognition in the Wild

Yunhao Liu1,   Lu Qi2,   Yu-Ju Tsai2,   Xiangtai Li3,   Kelvin C.K. Chan4,   
Ming-Hsuan Yang2,4,  
1Dalian University of Technology, 2The University of California, Merced, 3Nanyang Technology University, 4Google Research
Paper arXiv Video Code
empty


In real-world applications, face recognition systems frequently encounter probe images of low quality (LQ), which presents a significant domain gap compared to the high-quality (HQ) embedding gallery. Our method (c) addresses this challenge by integrating the features from the LQ images with those of enhanced HQ images in the fusion structure. Compared with conventional methods (a) (b), our method effectively bridges the domain gap, ensuring more accurate and reliable face recognition performance in real-world conditions.

Abstract

In this paper, we tackle the challenge of face recognition in the wild, where images often suffer from low quality and real-world distortions. Traditional heuristic approaches—either training models directly on these degraded images or their enhanced counterparts using face restoration techniques—have proven ineffective, primarily due to the degradation of facial features and the discrepancy in image domains. To overcome these issues, we propose an effective adapter for augmenting existing face recognition models trained on high-quality facial datasets. The key of our adapter is to process both the unrefined and the enhanced images by two similar structures where one is fixed and the other trainable. Such design can confer two benefits. First, the dual-input system minimizes the domain gap while providing varied perspectives for the face recognition model, where the enhanced image can be regarded as a complex non-linear transformation of the original one by the restoration model. Second, both two similar structures can be initialized by the pre-trained models without dropping the past knowledge. The extensive experiments in zero-shot settings show the effectiveness of our method by surpassing baselines of about 3%, 4%, and 7% in three datasets. Our code will be publicly available at https://github.com/liuyunhaozz/FaceAdapter/.

Video

Method

Interpolate start reference image.


Joint Face Recognition Framework with Dual-Input Processing. This architecture processes both low-quality (LQ) and restored high-quality (HQ) images, extracting them by two identical face recognition models. The Fusion Structure integrates feature sets before passing them to an Angular Margin Softmax function for loss computation, optimizing the network for enhanced recognition accuracy.


Results

1. Comparison to Arcface on image pair

Empty

Comparative analysis of face recognition methods on image pair. (a) illustrates the comparison results using different methods on an image pair of the same person, while (b) presents the results for an image pair of different people. Each image pair is accompanied by a cosine similarity score, ranging from -1 to 1, where a higher score indicates greater similarity between the images, and vice versa.



2. Comparison to Arcface on 1:N matching

Empty


Comparative analysis of face recognition methods on 1:N matching. This figure displays the Top 5 matching results for blurred face probes using both Arcface and our method. Each line contains an input probe and the Top 5 matches of the gallery using each of the two methods.

BibTeX

 @misc{liu2023effective,
        title={Effective Adapter for Face Recognition in the Wild}, 
        author={Yunhao Liu and Lu Qi and Yu-Ju Tsai and Xiangtai Li and Kelvin C. K. Chan and Ming-Hsuan Yang},
        year={2023},
        eprint={2312.01734},
        archivePrefix={arXiv},
        primaryClass={cs.CV}
  }