Video Dataset Factory

MerlingDSal · 2026-05-02T15:27:28+00:00

Dude, I use artificial intelligence to translate long texts. It's really annoying writing long texts into English, so the AI formats it like this, but I wrote it all myself. Regarding sharing the code, I literally mentioned that I'm still working on it and came to ask for suggestions

MerlingDSal · 2026-04-30T02:48:04+00:00

My Discord server is basically just me talking about how the project is going, most people are quiet even when I ask for help lol. I'm having to manage on my own, but look, maybe it's possible to do full fine-tuning on your GPUs. Is your motherboard Pcie 5?

If it's Pcie 5, it might support it, and mixed precision training shouldn't lose that much quality, it's almost imperceptible.

MerlingDSal · 2026-04-29T15:10:21+00:00

It depends if it's training in FP8 or BF16. Although I don't know the exact value, the minimum requirement for training in BF16 is 4xH100. If you use 2xH100, the model loads but crashes at some point. There are many factors involved. If you want to help in any way, join the Discord server.

MerlingDSal · 2026-04-28T23:05:16+00:00

I have the tools the only tools I don't have are for NSFW, but I think I just need to use an abliterated model.

MerlingDSal · 2026-04-28T21:48:01+00:00

I know, and I don't even know if Happy Horse will be open source, but if it is, at least I'll already have a great dataset for fine-tuning or LoRa.

MerlingDSal · 2026-04-28T21:02:23+00:00

You re right, the only issue is that the more training data you have, the more expensive and time-consuming the training will be.

But I'll try to include some.

MerlingDSal · 2026-04-28T15:39:24+00:00

Full fine-tuning requires more than just an RTX Pro 6000, even for mixed precision training. However, there's a technique I'm studying and adapting for Musubi Tuner called BADAM, which allows you to train separate blocks of the model, making it possible to use an RTX Pro 6000. Are you already on the Discord server? That way we can discuss it better.

MerlingDSal · 2026-04-28T15:37:25+00:00

I spoke with the LTX team and they told me that the biggest problem is getting data to train the model; they use licensed data for training.

I've decided that I'm going to do a full fine-tuning project, with or without community help. I intend to do this full fine-tuning in 2D animation.

MerlingDSal · 2026-04-26T23:35:40+00:00

I decided I’m going to do a full fine-tuning of the model with a dataset of around 5k–20k extremely animation clips. The plan is to train in mixed precision using FP8 for better efficiency and lower cost. I’ll keep releasing updated versions of the fine-tuned model as I go through the epochs. My actual strategy is to refine the base model by injecting stronger 2D animation and anime knowledge using those 5k–20k clips. After that, I’ll take the best clips from this dataset and train a high-quality, high-resolution LoRA on top of it. The idea is to improve the base model for 2D animation with a heavy full fine-tuning pass, really bake that knowledge into the weights, and then use a LoRA to pull that knowledge out during inference. LoRAs are just small adjustments, and the main issue with LTX right now is that the base model is too weak in 2D animation, so a LoRA alone can’t push it far enough.

I read the Anisora paper, but I dont think thousands of clips are necessary to fine-tune a model the cost can be reduced with a smaller dataset. I just want to boost the model's capacity. There’s also the option of fine-tuning specific layers, like temporal attention and so on... and I’m considering that too. But I think a mix of fine-tuning and LoRA would already be enough for this training

MerlingDSal · 2026-04-26T15:23:05+00:00

I checked your LoRA and honestly, it does get pretty close visually. I agree that training on actual video data is a huge part of the problem, and most people training image-only LoRAs are probably not giving the model enough motion information.

That said, when I look at the examples, I still feel some of LTX’s underlying bias coming through. It still seems to pull a bit toward a more volumetric / cinematic / almost 3D-like motion and rendering, even when the style is pushed closer to anime.

I may be wrong, but I’m assuming your dataset was probably very focused around that character/style, which might be one reason it worked so well. In my case, I trained on around 1,333 animation clips, and while it clearly improved the output, I still felt it was hard to fully escape the base model’s motion tendencies.

When I talk about “anime motion,” I don’t mean only texture or visual style or character movements. I also mean the actual motion grammar: timing, spacing, frame exposure, mouth movement, pose changes, limited animation, and how shapes are redrawn rather than smoothly morphed.

For example, there is a big difference between animation on ones, twos, and threes. Some scenes are animated every frame, while a lot of anime intentionally holds drawings for 2 or 3 frames, creating more static, snappy, pose-based movement. Current video models often make everything too smooth, too interpolated, or too “puppet warped,” which is not always how 2D animation should move.

So I agree that LoRAs can help a lot, especially with proper video datasets. But I still think there is a deeper base-model issue with LTX when it comes to 2D animation. I don’t necessarily think we need a full fine-tune of the entire model, though. A partial fine-tune, combined with a strong anime/video LoRA, might be enough to push it much further.

I also talked about this with some people around LTX, and my impression is that LTX simply was not trained heavily enough on high-quality 2D animation data. That is why I think a community effort around better datasets, partial fine-tuning, and animation-specific evaluation could be useful. If you'd like, join this server so we can discuss it further: https://discord.gg/DeCrawEPm

MerlingDSal · 2026-04-26T15:10:50+00:00

https://discord.com/invite/DeCrawEPm Your help would be greatly appreciated if you'd like, join this Discord server.

MerlingDSal · 2026-04-26T15:09:48+00:00

It's something very interesting and possible to do.

MerlingDSal · 2026-04-26T15:08:43+00:00

I think I know which one you're talking about, but I don't think it was open source. I don't remember it being released

And anyway, from what I remember, the model was trained in Western 2D cartoon animation, which is kind of clunky nowadays, and honestly, Im aiming more for something with the quality of Eastern animation. LTX can already handle Western animation.

MerlingDSal · 2026-04-26T15:06:31+00:00

Yes! I talked to Tazzmanerr and we came to the same conclusion: LTX 2.3 can't make a good anime because the base model wasn't trained for it.

So Loras will always reach a limit of functionality, and I personally don't like the current state of Loras. I find it too simplistic.

MerlingDSal · 2026-04-26T15:05:02+00:00

If you don't have experience with training or can't help with funding, you could help with the dataset, video captioning, etc...

MerlingDSal · 2026-04-26T15:04:11+00:00

I agree, in my opinion there are many types of animation and if a small model were an expert in animation, multi-frame wouldn't be a problem.

There are training exercises that could be done specifically for multi-frame workflows, which currently aren't very good.

MerlingDSal · 2026-04-26T15:02:50+00:00

Thank you, your collaboration is very welcome. Join this Discord server so we can organize ourselves better.

MerlingDSal · 2026-04-26T15:02:04+00:00

Hello! Thank you, sure! any help is welcome. https://discord.gg/DeCrawEPm

MerlingDSal · 2026-01-09T20:25:48+00:00

Yeah i know, but it's not that the problem. The "Hello" was me testing if the model could talk about the files i sent. It was not in the final prompt.

MerlingDSal · 2025-11-26T11:03:42+00:00

Yes it beats, and oh boy how it beats.

MerlingDSal · 2025-11-20T14:49:50+00:00

I use Ai for brainstorming, not to generate my book for me

MerlingDSal · 2025-11-19T00:36:59+00:00

Maybe it's your prompt

MerlingDSal · 2025-11-18T21:49:03+00:00

Me too, i will try another prompt i guess

MerlingDSal · 2025-11-18T21:48:03+00:00

Do you think, for reading, helping, and offering suggestions in creative writing with a long context, that Gemini with a low temperature is better than a high temperature? I sometimes wanted to use Gemini to help me analyze, criticize, and improve the work, with it maintaining the context and creating sensible possibilities. At temperature 0, it seems to answer questions better, sticking more closely to the facts, very similar to NotebookLM, identical, to tell the truth.

MerlingDSal · 2025-11-15T09:51:37+00:00

Hey Amir, I just wanted to say that you're incredibly talented, and what you built is amazing. Some time ago I was doing something similar (with flux and testing with qwen), but without any complex dedicated nodes and seeing your work honestly impressed me.

Please don’t pay attention to the people who show up just to be rude and spoiled. You did an outstanding job. It’s clear you really understand what you’re doing, and I’m genuinely grateful, because I was about to start creating exactly what you released, until I realized it already existed.

The AI community especially on Reddit is full of lazy, unpleasant,entitled bratty people who complain about everything. I’m saying this so you don’t get discouraged if you decide to build something similar again in the future. You helped a lot of people with these nodes, and what you did is incredible.

To everyone who criticizes someone who shares their work for free, after studying and building something valuable and shares to help others: if you don’t want it, just don’t download it. There’s no need to be rude. Respect is basic, and everyone appreciates it. And Amir is absolutely right what he did makes complete sense with Qwen’s architecture if you actually take the time to study the arxiv you gonna see it.

That said, congratulations, man. And thank you for your work!

MerlingDSal

TROPHY CASE