Seeing the Sound: Multilingual Lip Sync for Real-Time Face-to-Face Translation by Amirkia Rafiei Oskooei, Mehmet S. Aktaş, Mustafa Keleş

“…Our research tackles the primary challenge of achieving accurate lip synchronization across culturally diverse languages, filling a significant gap in the literature by evaluating the generalizability of lip sync models beyond English. Specifically, we develop a novel evaluation framework combining quantitative lip sync error metrics and qualitative assessments by human observers. …”
Get full text

Multi-Level Feature Dynamic Fusion Neural Radiance Fields for Audio-Driven Talking Head Generation by Wenchao Song, Qiong Liu, Yanchao Liu, Pengzhou Zhang, Juan Cao

“…Compared with previous methods, it performs better in terms of image quality, lip sync, and other aspects.…”
Get full text