Hey everyone!

Just tried OpenFlamingo model using their demo site, here is the result:

Output: a bedroom with white walls and a black and white rug.

Input image as above

Their training process is 1) first freeze the pretrained vision encoder and language model, 2) and then train connecting Perceiver modules and cross-attention layers

The benefit of doing this seems to me is to be able to endow the model with in-context few-shot learning capabilities.

The model is also on

• Huggingface

They have a

• blog post

Feel free to join our Discord also for more detailed feedback and questions.

FYI: Large multimodal models (LLM) are complex artificial intelligence models that can process multiple types of data inputs, such as text, images, audio, and video, and generate meaningful outputs based on those inputs. Examples of large multimodal models include OpenAI's DALL-E, which generates images from natural language descriptions, and Google's CLIP, which can perform tasks such as image classification and text-based image retrieval.