← All publications

Conference · 2026

Video and Scene Understanding

MReactor: Offline Multiple Appropriate Facial Reaction Generation with Hierarchical Cognitive Disentanglement

Jiachen Luo, Jiajun He, Shuai Shen, Lin Wang, Huy Phan, Joshua Reiss, Lin Haijun, Bjoern Schuller, Zeyu Fu, Siyang Song

CVPR 2026 Findings

Pipeline of MReactor: multi-modal behaviour perception, hierarchical cognition disentanglement and multiple facial reaction generation.

Abstract

Overview

Facial reactions are central to dyadic communication, revealing how listeners perceive and respond to speakers' behaviours. Unlike traditional facial reaction generation methods that assume a one-to-one mapping, human reactions are inherently one-to-many, motivating the Multiple Appropriate Facial Reaction Generation (MAFRG) task. Despite the significant advances achieved in recent years, existing approaches still face challenges in noisy multi-modal perception, entangled local and global dynamics, and limited semantic controllability. To trackle these problems, we present MReactor in this work, which is the first offline and cognitively grounded MAFRG framework inspired by the Human Model Processor. Specifically, the MReactor integrates (i) multi-modal behaviour perception for stable cue extraction, (ii) hierarchical cognition disentanglement for fine- and coarse-grained cues, and (iii) multiple appropriate generation for diverse and contextually coherent reactions via contextual behaviour and recovery expression. Extensive experiments on the MARS dataset show that our MReactor achieved state-of-the-art results in synchrony, appropriateness and diversity, producing natural and human-like facial reactions.

Citation

BibTeX

@inproceedings{mreactor,
  title = {MReactor: Offline Multiple Appropriate Facial Reaction Generation with Hierarchical Cognitive Disentanglement},
  author = {Jiachen Luo and Jiajun He and Shuai Shen and Lin Wang and Huy Phan and Joshua Reiss and Lin Haijun and Bjoern Schuller and Zeyu Fu and Siyang Song},
  year = {2026},
  booktitle = {CVPR 2026 Findings}
}