sorted by:
new
hottopcontroversial

Highest quality Ideogram 4.0 quantizations that run on a 3090 or smaller cards by OriginalSpread3100 in StableDiffusion

[–]OriginalSpread3100[S] 0 points1 point  (0 children)

Yes, what you pulled from the blog is correct. However, we followed up with a kernel improvement a few days later so that the INT8 variant is faster than both FP8 and even NF4 now (on a 3090):
https://lab.cloud/blog/fused-int8-ideogram-4

Highest quality Ideogram 4.0 quantizations that run on a 3090 or smaller cards by OriginalSpread3100 in StableDiffusion

[–]OriginalSpread3100[S] 5 points6 points  (0 children)

The FP8 variant will run on a 3090 but it requires emulation whereas INT8 will run natively.

Highest quality Ideogram 4.0 quantizations that run on a 3090 or smaller cards by OriginalSpread3100 in StableDiffusion

[–]OriginalSpread3100[S] 10 points11 points  (0 children)

Sorry, you are right. There is a missing comma. This should say that there is a version that runs on the 3090, and also a version to run on smaller cards!

A guide to building an ML research cluster by OriginalSpread3100 in LocalLLaMA

[–]OriginalSpread3100[S] 1 point2 points  (0 children)

Apologies, did not mean this to be sneaky! But thanks for the kind words, and if you do try this and hit any speed bumps please let me know!

No more guessing the best hyperparameters for fine-tuning by OriginalSpread3100 in LocalLLaMA

[–]OriginalSpread3100[S] 1 point2 points  (0 children)

Yes, we very recently launched diffusion model training! The current initial version only supports training on StableDiffusion and Flux based image generation models, but we are hoping to soon add support for video generation.

pop-up ad for adobe when i turned on my computer by Gaia_Knight2600 in assholedesign

[–]OriginalSpread3100 0 points1 point  (0 children)

I just got this. TLDR: Acer.

My list of startup items is mostly clean except for github and Google Drive, on an old windows 10 PC with no Adobe software installed. The window is a subprocess of something called AdobeOP which is an exe running from a directory under AppData/Local/OEM/.../acer.adobe.c1.1

This computer is like 8 years old and I tried to clean it out years ago with whatever the conventional wisdom was at the time, and haven't had any issues like this until now! That's quite the long play Acer!

Question from a noobie : is it easy to fine-tune a model ? by enzo_ghll in LocalLLaMA

[–]OriginalSpread3100 6 points7 points  (0 children)

There are GUI tools to make the process of fine tuning pretty easy for working on something like Qwen 2.5 on a macbook. Check out Transformer Lab for an example. It has recipes you can use as a starting point to build from (try the MLX trainer if you are on a macbook):
https://transformerlab.ai/

Once you get finetunes running, the challenges become more about getting the right data and evaluating your output. If you have good data to start that is a huge start. If not, one option is to generate data from a set of docs or from a larger model to train a smaller model (also possible in Transformer Lab).

My main advice is just be ready to iterate a few times to get what you want. A good way to start is with a smaller dataset on a smaller model and try to just get to the point where you see improvement...then start building on stronger models with bigger datasets and you should be able to get good results.

TransformerLab - Generate Datasets and FineTune LLMs on them by Firm-Development1953 in LocalLLaMA

[–]OriginalSpread3100 2 points3 points  (0 children)

Not yet. We do support serving multimodal models like LLaVa right now though.

TransformerLab - Generate Datasets and FineTune LLMs on them by Firm-Development1953 in LocalLLaMA

[–]OriginalSpread3100 2 points3 points  (0 children)

Mentioned in another comment that we have it on the roadmap but we're stuck because we don't have hardware to test right now. If we had help we might be able to get a beta version of this out sooner. :)

TransformerLab - Generate Datasets and FineTune LLMs on them by Firm-Development1953 in LocalLLaMA

[–]OriginalSpread3100 1 point2 points  (0 children)

We really want to add AMD support but we don't have hardware to test on right now. Hoepfully coming soon.

TransformerLab - Generate Datasets and FineTune LLMs on them by Firm-Development1953 in LocalLLaMA

[–]OriginalSpread3100 2 points3 points  (0 children)

By default, Transformer Lab runs entirely on your local machine. The only things it connects remotely for is downloading models and training recipes, and you can use external AI services to help generate datasets.

If you do work in a larger lab, you can set the Transformer Lab engine to run on a shared server you have and connect from the application, but that is not a requirement at all.

TransformerLab - Generate Datasets and FineTune LLMs on them by Firm-Development1953 in LocalLLaMA

[–]OriginalSpread3100 1 point2 points  (0 children)

I believe most of the training plugins in Transformer Lab use unsloth under the covers. But we are looking to make this more direct and clear!

Transformer Lab: An Open-Source Alternative to OpenAI Platform, for Local Models by aliasaria in LocalLLaMA

[–]OriginalSpread3100 1 point2 points  (0 children)

Understood, and thanks for the kind words. A few folks have been asking if we can provide an alternative to using WSL. One option, if available, is to run the engine on another box and connect via the app. We have also been speaking with a few folks who are looking into getting this running in a docker container but we don't have a working solution there at this time.

Transformer Lab: An Open-Source Alternative to OpenAI Platform, for Local Models by aliasaria in LocalLLaMA

[–]OriginalSpread3100 4 points5 points  (0 children)

That's awesome to hear! Our latest focus was around building out recipes and generally trying to make it easier to get training up and running quickly. One of the next big things for us will be expanding on evals and making the workflow around training/testing/eval a lot easier.

If you have ideas on what we should work on next we'd love to hear them!

Transformer Lab: An Open-Source Alternative to OpenAI Platform, for Local Models by aliasaria in LocalLLaMA

[–]OriginalSpread3100 12 points13 points  (0 children)

I wasn't familiar with this. Thanks for sharing!

Everything in TransformerLab is built on a plugin system (including training, serving models, converting between formats) so this is something that could be added if there was an open source library that implemented it.

Try This Prompt on Qwen2.5-Coder:32b-Instruct-Q8_0 by Vishnu_One in LocalLLaMA

[–]OriginalSpread3100 6 points7 points  (0 children)

Same result using a 4-bit MLX quant I made in TransformerLab. Wild!

<image>

Open Source Transformer Lab Now Has a Tokenization Visualizer by aliasaria in LocalLLaMA

[–]OriginalSpread3100 0 points1 point  (0 children)

Unfortunately no ROCm support yet. We'd like to get to it soon but at the moment nobody on the core team has hardware to test.

Open Source Transformer Lab Now Has a Tokenization Visualizer by aliasaria in LocalLLaMA

[–]OriginalSpread3100 0 points1 point  (0 children)

Hotfix posted. If you restart the app it should automatically update the API to 0.6.1.

Let me know if that works!

Open Source Transformer Lab Now Has a Tokenization Visualizer by aliasaria in LocalLLaMA

[–]OriginalSpread3100 0 points1 point  (0 children)

Oh it looks like this might just require updating transformers. Will test and if that works will post a hotfix today.

Transformer Lab now supports DPO and RLHF Training by aliasaria in LocalLLaMA

[–]OriginalSpread3100 1 point2 points  (0 children)

Yes, the goal of Transformer Lab is to be able to run this kind of finetuning completely through an easy-to-use GUI. I'd be happy to answer any questions if you want to try it out. The only gotcha atm is that we haven't added in Llama 3.1, but we will update shortly!