November 11, 2024
SeedEdit
Align Image Re-Generation to Image Editing
We present SeedEdit, a large diffusion model for revising images based on any text prompts. It optimally balances image reconstruction and image re-generation, which is achieved by progressively aligning an image generator to a strong image editor. SeedEdit achieves impressive zero-shot stable editing of high aesthetic/resolution images, and enables sequential revisions of images.
Method
The core difficulty of the image editing problem is the scarcity of pairwise image data. To address this problem, we regard text-to-image (T2I) generation model as a weak editing model, which achieves "editing" by generating a new image with a new prompt. We then distill and align it into a image-conditioned editing model.
We propose an effective editing data generation and filtering strategy, which is able to progressively align any T2I model to a strong image editor.
We design a novel editing architecture with precise editing instruction interpretation and image generation.
Built on our Seed T2I foundational model, SeedEdit delivers stable, high-aesthetic image edits which maintain image quality through unlimited rounds of editing instructions.
Architecture
We introduce causal diffusion model for image-to-image generation. Two branches with shared parameters are applied to the input and output images/texts, respectively.
Results
SeedEdit can conduct diverse types of image editing such as local replacement, geometric transform, relighting, style change or a mixture of them with good image quality. Click on the images to see!
Let it be flying over the ocean
Close his eyes and smile
The house is above the sky, fantasy style
Make it a wizard
Change the words to “cheap price”
Studio light from left side
Empty street in a quite night
Let she look to her right
Replace the rabbit with a fawn
The images and audios used in these demos are from public sources. If there are any concerns, please contact us (doubao-llm@bytedance.com) and we will delete it in time.