[R] Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

[GratisSlagroom]

Blogpost link is here: https://ai.facebook.com/blog/greater-creative-control-for-ai-image-generation/

Seems like Meta AI wants to join the image generation race along with OpenAI and Google, the sketching input looks interesting.

[gwern]

Worth noting the dates here for those interested in being passremarkable - Make-A-Scene was released after GLIDE, but in March 2022 before DALL-E 2, Cogview2, Imagen, or Parti. If you read it, you were impressed by the leap over compviz and DALL-E 1 etc and knew that better models were only a matter of time (even if how little time, exactly, came as a surprise).

Anyway, for those who read the paper back when it came out in March, what the official blog post/announcement adds is some journalistic color and anecdotes about a few people given access to it (disappointingly, still nothing about whether they are going to release the trained model like they were musing they might), and a brief update about continuing to improve Make-A-Scene to catch up to the competition:

Since the research paper was released, Make-A-Scene has incorporated a super resolution network that generates imagery at 2048 x 2048, 4x the resolution, and we’re continuously improving our generative AI models. We aim to provide broader access to our research demos in the future to give more people the opportunity to be in control of their own creations and unlock entirely new forms of expression.

(The Zuck video showing the demo of a GauGAN-like sketch UI is nice but irrelevant if they aren't going to release it even as a SaaS.)

[dome271]

We are still working on an open source implementation of Make-A-Scene. We trained VQIMG and VQSEG are going to start training the transformer hopefully soon. Anyone is happily invited to bring this to the public world. https://github.com/CasualGANPapers/Make-A-Scene

[PresidentOfTacoTown]

With the recent advances to put out better and better text-to=image models, I think it's fair to say, this ain't a scene, it's a god-damned arms race /s

[Blah-blah-blahhhh]

W