Ziyan Chen

In recent years, Face Image Restoration (FIR) has achieved remarkable progress by leveraging generative models to reconstruct high-quality and identity consistent facial images from degraded inputs. However, when extended to Face Video Restoration (FVR), these methods often struggle to maintain distinctive facial characteristics, leading to noticeable identity drift across frames. To address this limitation, concurrent studies have introduced few-shot personalized LoRA finetuning. Yet, this approach requires substantial re-optimization when applied to new identities, and existing reference-guided mechanisms remain insensitive to fine-grained identity cues. To overcome these challenges, we present RestoreAnyone, a zero-shot identity-consistent FVR framework that enforces fine-grained identity preservation throughout the generative process. We systematically explore a broad spectrum of integration strategies and identify an optimal design that maintains identity consistency across both structural and fine-detail levels. Furthermore, to address the scarcity of large-scale video data, we construct MovieID, a comprehensive movie-level reference-based FVR benchmark comprising 32k high-quality video clips of 2.9k distinct identities captured across diverse scenes. Extensive experiments on this benchmark demonstrate that our framework achieves state-of-the-art performance, improving video identity consistency by 36% and delivering substantially higher visual fidelity in restored face videos.

We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks in a unified framework. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. Each stage is developed independently but they work seamlessly in a cascaded manner. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details. Specifically, IRControlNet is trained based on specially produced condition images without distracting noisy content for stable generation performance. Moreover, we design a region-adaptive restoration guidance that can modify the denoising process during inference without model re-training, allowing users to balance quality and fidelity through a tunable guidance scale. Extensive experiments have demonstrated DiffBIR’s superiority over state-of-the-art approaches for blind image super-resolution, blind face restoration and blind image denoising tasks on both synthetic and real-world datasets. The code is available at https://github.com/XPixelGroup/DiffBIR.

Blind image face restoration (BFR) has significantly progressed over the last several years while the real-world video face restoration (VFR) which is more challenging for more complex face motions such as moving gaze directions and facial orientations involved remains unsolved. Typical BFR methods are evaluated on privately synthesized datasets or self-collected real-world low-quality face images which are limited in their coverage of real-world video frames. In this work we introduce a new benchmark dataset named FOS with a taxonomy of Full Occluded and Side faces from both video frames and images with not only synthetic and real-world degradations covered but also rich facial data contents such as various expressions poses and ethnicity involved. Given the established datasets we benchmark both the state-of-the-art BFR methods and the video super resolution (VSR) methods to comprehensively study current approaches revealing their potential and limitations in VFR tasks. For performance assessment we not only employ the commonly used general image quality assessment (IQA) metrics but also explore face IQA metrics by leveraging an elaborately designed user study. With extensive experimental results and detailed analysis provided we gain insights from the successes and failures of current methods. In addition these studies also pose challenges in current BFR approaches and shed light on future VFR research.

Compared to query-based black-box attacks, transfer-based black-box attacks do not require any information of the attacked models, which ensures their secrecy. However, most existing transfer-based approaches rely on ensembling multiple models to boost the attack transferability, which is time-and resource-intensive, not to mention the difficulty of obtaining diverse models on the same task. To address this limitation, in this work, we focus on the single-model transfer-based black-box attack on object detection, utilizing only one model to achieve a high-transferability adversarial attack on multiple black-box detectors. Specifically, we first make observations on the patch optimization process of the existing method and propose an enhanced attack framework by slightly adjusting its training strategies. Then, we analogize patch optimization with regular model optimization, proposing a series of self-ensemble approaches on the input data, the attacked model, and the adversarial patch to efficiently make use of the limited information and prevent the patch from overfitting. The experimental results show that the proposed framework can be applied with multiple classical base attack methods (eg, PGD and MIM) to greatly improve the black-box transferability of the well-optimized patch on multiple mainstream detectors, meanwhile boosting white-box performance.