We introduce a general diffusion synchronization framework for generating diverse visual content, including ambiguous images, panorama images, 3D mesh textures, and 3D Gaussian splats textures, using a pretrained image diffusion model. We first present an analysis of various scenarios for synchronizing multiple diffusion processes through a canonical space. Based on the analysis, we introduce a novel synchronized diffusion method, SyncTweedies, which averages the outputs of Tweedie’s formula while conducting denoising in multiple instance spaces. Compared to previous work that achieves synchronization through finetuning, SyncTweedies is a zero-shot method that does not require any finetuning, preserving the rich prior of diffusion models trained on Internet-scale image datasets without overfitting to specific domains. We verify that SyncTweedies offers the broadest applicability to diverse applications and superior performance compared to the previous state-of-the-art for each application.
âž¡
âž¡
âž¡
âž¡
âž¡
âž¡
@article{Kim2024SyncTweedies,
title = {SyncTweedies: A General Generative Framework Based on Synchronized Diffusions},
author = {Kim, Jaihoon and Koo, Juil and Yeo, Kyeongmin and Sung, Minhyuk},
year = {2024},
journal = {arXiv:2403.14370},
}