We introduce a general framework for generating diverse visual content, including ambiguous images, panorama images, mesh textures, and Gaussian splat textures, by synchronizing multiple diffusion processes. We present exhaustive investigation into all possible scenarios for synchronizing multiple diffusion processes through a canonical space and analyze their characteristics across applications. In doing so, we reveal a previously unexplored case: averaging the outputs of Tweedie's formula while conducting denoising in multiple instance spaces. This case also provides the best quality with the widest applicability to downstream tasks. We name this case SyncTweedies. In our experiments generating visual content aforementioned, we demonstrate the superior quality of generation by SyncTweedies compared to other synchronization methods, optimization-based and iterative-update-based methods.
"A dumpster"
"A clutch bag"
"A lemon"
"A hand carved wood turtle"
"A nascar"
"A hamburger"
"An hourglass"
"A jeep"
"A turtle"
âž¡
"A golden statue
of a turtle"
"A car"
âž¡
"A luxurious
red sports car"
"A lantern"
âž¡
"A chinese style lantern"
"A nascar"
âž¡
"A car with graffiti"
"An elephant"
âž¡
"An african elephant"
"An axe"
âž¡
"A wooden axe"
"A majestic red chair"
"A photo of cucumbers"
"A photo of a yellow excavator covered in snow"
"A photo of a white cruise ship at sea"
"A leather chair"
"A photo of corns"
"A white drum kit"
"A photo of a pirate ship at sea"
"A photo of a mountain range at twilight"
"A photo of a beautiful ocean with coral reef"
"A photo of a lake under the northern lights"
"A house at night"
"An old looking library"
"A room that has been painted gold"
Case1
Case2
(SyncTweedies)
Case3
Case4
Case5
Paint-it
TEXTure
Text2Tex
"Baseball glove"
Case1
Case2
(SyncTweedies)
Case3
Case4
Case5
Paint-it
TEXTure
Text2Tex
"Minivan"
Case1
Case2
(SyncTweedies)
Case3
Case4
Case5
Paint-it
TEXTure
Text2Tex
"iPod"
Case1
Case2
(SyncTweedies)
Case3
Case4
Case5
Paint-it
TEXTure
Text2Tex
"Pigeon"
Case2
(SyncTweedies)
Case5
SDS
IN2N
"A photo of a tree with multicolored leaves"
Case2
(SyncTweedies)
Case5
SDS
IN2N
"A photo of a wooden carving of a microphone"
Case2
(SyncTweedies)
Case5
SDS
IN2N
"A photo of an intricate wooden carving of a ship"
Case2
(SyncTweedies)
Case5
SDS
IN2N
"A photo of a purple chair"
Case2
(SyncTweedies)
Case5
SDS
IN2N
"A photo of carrots"
Case2
(SyncTweedies)
Case5
SDS
IN2N
"A photo of a tree covered in snow"
@article{Kim2024SyncTweedies,
title = {SyncTweedies: A General Generative Framework Based on Synchronized Diffusions},
author = {Kim, Jaihoon and Koo, Juil and Yeo, Kyeongmin and Sung, Minhyuk},
year = {2024},
journal = {arXiv:2403.14370},
}