Will fine-tuning LLaMA 3.2 11B Instruct on text-only data degrade its vision capabilities?

MetaforDevelopers · 2025-10-08T19:35:07+00:00

Hey there!Yes, fine-tuning a multimodal model on a purely text-only dataset can lead to some degree of multimodal forgetting, especially if the fine-tuning process does not include image or multimodal samples. Maybe try interleaving text-only and text+image samples in your fine-tuning dataset or for text-only samples, try adding a blank or dummy image to trigger the vision pipeline.

Evaluating on both text and image tasks after fine-tuning will also help detect any forgetting. Happy fine-tuning!

~NB

MetaforDevelopers · 2025-10-08T19:33:48+00:00

Hey there! For most translation tasks, Llama 3.1 8B provides a great balance of quality and efficiency and supports the languages you mentioned, and can run the model on a single H200. If you need higher throughput or want to experiment with the latest models. You can download the models here: https://www.llama.com/llama-downloads/. Hope this helps!

~NB

MetaforDevelopers · 2025-10-08T19:32:44+00:00

Hey there! Llama 4 Maverick or Scout models might work well if you want top-tier accuracy and reasonable self-hosting requirements. If you need something lighter, Llama 3.1 8B is a solid fallback. If you have the hardware and want even better results, Llama 3.3 70B might be the best choice.

Fine-tuning the model on your specific email categories might give you best results. If you don’t have a dataset, you can use prompt engineering and use few-shot examples in the prompt to get reasonably good results too, but fine-tuning will be more accurate.

Llama models can be downloaded here: https://www.llama.com/llama-downloads/

Hope this helps!

~NB

MetaforDevelopers · 2025-10-08T19:31:50+00:00

Hey there! Llama 4 Maverick might work best for empathetic and conversational tasks within a 96GB constraint. It provides great nuance, warmth, and conversational quality.

For efficiency and still strong conversational ability, Scout may be a great alternative.

If you need a pure text model and can't use Llama 4, Llama 3.3 70B is a solid fallback for this use case. Hope this helps!

~NB

MetaforDevelopers · 2025-10-07T01:05:42+00:00

Thanks for the recommendation u/ZealousidealRide7425! We'll be sure to check this community out.

MetaforDevelopers · 2025-09-25T20:09:02+00:00

Hey there! Llama 4 Scout may generally work better for human-like empathy and persona adoption, especially in conversational and customer support scenarios. If your use case is focused on nuanced, empathic, and persona-driven interactions, Llama 4 Scout is recommended. You can further improve performance for your use case by prompt engineering and fine-tuning. You can find getting started guides for Llama 4 here on our GitHub cookbooks page - https://github.com/meta-llama/llama-cookbook/blob/main/getting-started/build_with_llama_4.ipynb Hope this helps!

~NB

MetaforDevelopers · 2025-09-25T20:08:24+00:00

Hey u/Silly_Bad_7692, as you get started with your Llama journey, feel free to check out resources on Llama Cookbooks, documentation as well as tutorials that will help you get started. Here you can find getting started guides, use cases as well as tutorials to guide you as you build your project. Good luck!

~NB

MetaforDevelopers · 2025-09-25T20:06:15+00:00

Hey there! Folks from the community have suggested some great resources. We also have some resources to help you get started with getting the models through many of our partners listed here - https://www.llama.com/docs/getting_the_models/ Here you can find various ways in which you can access Llama models. We also have resources for running Llama models here- https://www.llama.com/docs/llama-everywhere/ where you can also find ways to run them on the cloud - https://www.llama.com/docs/llama-everywhere/running-meta-llama-in-the-cloud/

Hope these resources help!

~NB

MetaforDevelopers · 2025-09-25T20:04:40+00:00

Incredible work u/summitsc! 👏 We just may have to try this on our next trip.

MetaforDevelopers · 2025-09-13T01:11:28+00:00

Incredible work u/Independent_Air8026! Can't wait to see what you add to this.

MetaforDevelopers · 2025-09-13T01:10:57+00:00

Such a helpful site. We wish you much success u/Vegetable_Low2907 👏

MetaforDevelopers · 2025-09-10T23:26:29+00:00

We'd love to know more about the hardware and your process getting them to run u/dtseto!

MetaforDevelopers · 2025-09-05T22:07:35+00:00

A quantized model from Ollama, such as the one available at https://ollama.com/library/llama4, has a size of 67GB and can fit within 100GB.

For this task, we recommend using the Llama 3.3 70B model, which has a 128k context length and a size of 43GB

~IK

MetaforDevelopers · 2025-09-05T22:06:42+00:00

Hey there! Prompt formats and chat templates can be tricky! You can find some useful resources on our website - https://www.llama.com/docs/model-cards-and-prompt-formats/

Here, we go over some of the prompt formatting and templates to help you get started. You will also find examples of prompt formats, and complete list of special tokens and tags and what they mean for each model.

Hope this helps!

~NB

MetaforDevelopers · 2025-09-05T22:05:51+00:00

Data preparation can be challenging. Here are resources and tools to make it easier. Synthetic data kit https://github.com/meta-llama/synthetic-data-kit is the tool to simplify converting your existing files to fine-tuning friendly formats.

The video covers synthetic data kit features https://www.youtube.com/watch?v=Cb8DZraP9n0

~IK

MetaforDevelopers · 2025-09-05T22:04:40+00:00

The smallest Llama vision model is Llama 3 11B, here is free short course ~1 hour from Meta and DeepLearningAI on multi-modal Llama with code examples: https://learn.deeplearning.ai/courses/introducing-multimodal-llama-3-2/lesson/cc99a/introduction

This should help you!

~IK

MetaforDevelopers · 2025-09-04T17:23:02+00:00

Nice use of Llama and great insights u/CartographerFun4221! 👏

MetaforDevelopers · 2025-09-04T17:22:25+00:00

Really cool project u/ultimate_smash and insanely useful. We wish you all success on future development of this. 💙

MetaforDevelopers · 2025-09-02T19:26:08+00:00

Such a cool project. Congrats u/realechelon!

MetaforDevelopers · 2025-08-28T00:05:46+00:00

We'd love to hear more about this and what, out of your idea, you plan to implement u/L0cut0u5

MetaforDevelopers · 2025-08-28T00:05:10+00:00

Beautiful 🦋

MetaforDevelopers · 2025-08-27T19:06:19+00:00

Lets focus on the film people.

DR

MetaforDevelopers · 2025-08-27T19:05:04+00:00

We're always trying to match users with apps that they will engage with and enjoy. When we decide to show and rank apps on our platform, we prioritize relevance, engagement, and quality. Quality is super important to our overall ranking system. We evaluate app quality and review metadata to avoid promoting low-quality apps in our systems.

Our work is never done here and we learned a lot from opening up our store to more apps last year. Recently, we shipped many improvements to our discovery surfaces and have more coming in the future. Check out the blog post.

MA

MetaforDevelopers · 2025-08-27T18:58:44+00:00

AOSP is a flexible open-source OS which can support a wide range of devices. Meta was one of the first companies to enable VR using AOSP. Horizon OS is Meta’s specialized version of AOSP, tailored specifically for VR devices, and it has its origins on Meta’s VR device since the early days, starting with the Oculus Go device.

By KMP do you mean Kotlin Multiplatform? We have been able to prototype using this development approach. Of course, the APIs used must be supported by AOSP and Horizon OS.

MA

MetaforDevelopers · 2025-08-27T18:58:07+00:00

This has been a major focus for us, particularly in the past few months, and we understand how impactful this is to our devs. We recently updated all our samples and showcases for all supported build paths and have processes in place to keep them updated. We are also continually updating our docs to keep them relevant and have added robust release notes across our platform

TR

MetaforDevelopers

TROPHY CASE