3D rendering of dynamic face captures is a challenging problem, and it demands improvements on several fronts$\unicode{x2014}$photorealism, efficiency, compatibility, and configurability. We present a novel representation that enables high-quality volumetric rendering of an actor's dynamic facial performances with minimal compute and memory footprint. It runs natively on commodity graphics soft- and hardware, and allows for a graceful trade-off between quality and efficiency. Our method utilizes recent advances in neural rendering, particularly learning discrete radiance manifolds to sparsely sample the scene to model volumetric effects. We achieve efficient modeling by learning a single set of manifolds for the entire dynamic sequence, while implicitly modeling appearance changes as temporal canonical texture. We export a single layered mesh and view-independent RGBA texture video that is compatible with legacy graphics renderers without additional ML integration. We demonstrate our method by rendering dynamic face captures of real actors in a game engine, at comparable photorealism to state-of-the-art neural rendering techniques at previously unseen frame rates.

中文翻译：

---

FaceFolds：用于动态面的高效体积渲染的网格辐射流形

动态面部捕捉的 3D 渲染是一个具有挑战性的问题，它需要在多个方面进行改进$\unicode{x2014}$照片真实感、效率、兼容性和可配置性。我们提出了一种新颖的表示方法，可以以最少的计算和内存占用对演员的动态面部表演进行高质量的体积渲染。它在商品图形软件和硬件上本地运行，并允许在质量和效率之间进行优雅的权衡。我们的方法利用神经渲染的最新进展，特别是学习离散辐射流形来对场景进行稀疏采样以建模体积效果。我们通过学习整个动态序列的一组流形来实现高效建模，同时将外观变化隐式建模为时间规范纹理。我们导出单层网格和独立于视图的 RGBA 纹理视频，该视频与传统图形渲染器兼容，无需额外的 ML 集成。我们通过在游戏引擎中渲染真实演员的动态面部捕捉来演示我们的方法，其逼真度可与最先进的神经渲染技术以以前未见过的帧速率相媲美。