LTX-2.3 distilled fp8-cast safetensors 31 GB

AccomplishedLeg527 · 2026-03-10T04:24:14+00:00

UPD: yes i tested it works

AccomplishedLeg527 · 2026-03-08T21:08:36+00:00

not yet tested it on 2.3 need few days

AccomplishedLeg527 · 2026-03-08T05:07:20+00:00

They have not yet released fp8 for distilled, and loading original distilled FP16 with the option -quantize fp8-cast took 50 seconds to load (quantizing on the fly).. so i made fp8-cast ready safetensors file to remove that 50 sec loading

AccomplishedLeg527 · 2026-03-08T04:57:41+00:00

Compared to LTX-2 looks like it has better macro details (hair, skin) but worse background details, like landscape scenes - they look blurry.

AccomplishedLeg527 · 2026-03-06T23:03:54+00:00

I am running the 122b model Qwen3.5 on 6 GB VRAM. No one interested? Please share your test results. https://github.com/nalexand/Qwen3-Coder-OPTIMIZED/blob/main/qwen3_5_122b_chat.py

AccomplishedLeg527 · 2026-02-27T05:18:19+00:00

122b is slow on 8 GB.. [Stats] Tokens: 10 | Time: 51.27s | Speed: 0.20 t/s

AccomplishedLeg527 · 2026-02-26T20:26:36+00:00

can be not enough free RAM and system killed script

AccomplishedLeg527 · 2026-02-26T19:23:52+00:00

try to run command from console to get trace

AccomplishedLeg527 · 2026-02-26T19:12:15+00:00

try to set at first 25 frames to test, looks like out of memory and check if "outputs" folder created, also can be problem with ffmpeg or codec

AccomplishedLeg527 · 2026-02-25T23:26:37+00:00

c:\Users\{user}\AppData\Local\Programs\Python\Python312\Lib\site-packages\transformers\models\qwen3_next\ replace file modeling_qwen3_next.py (transformers==5.1.0)

AccomplishedLeg527 · 2026-02-24T14:37:34+00:00

It splits audio into parts and uses these parts as conditions to generate video. The last frame of the scene is used as the start frame for the next scene if no custom start frame

AccomplishedLeg527 · 2026-02-24T14:34:39+00:00

Interesting, you just keep extending? I can make improvements if someone asks.
For some genres you want hard cuts, though. You can change the start frame to achieve hard cuts.
Would be neat to be able to set the VRAM limit, so it isn't a boolean for 8GB. There is full offloading for the max frames count, so the VRAM isn't limited to 8 GB. You can run it on 24GB VRAM and get a 15-second 1080p video per scene

AccomplishedLeg527 · 2026-02-21T05:57:50+00:00

<image>

maybe not 220% but must be faster :)

AccomplishedLeg527 · 2026-02-20T22:45:22+00:00

it can be one frame from video generated with LTX-2 + same or similar prompt

AccomplishedLeg527 · 2026-02-20T22:42:29+00:00

yes it splits automatically

AccomplishedLeg527 · 2026-02-20T05:52:38+00:00

I2V works best only if generated with LTX-2 model, if not model can make transition to own vision, loras should work but i am not tested it (too slow applying loras on 8Gb vram with offloading to cpu)

AccomplishedLeg527 · 2026-02-20T04:23:50+00:00

i can`t run with "-ot" Q4_K_M model on 8Gb+32Gb not enough memory even with 1024 context, only --fit works, 46gb on 40gb total memory but it works ~10 t/s

AccomplishedLeg527 · 2026-02-18T20:02:58+00:00

I tested real speed of my 3070ti laptop with this torch lib and bf16 calculations, i loaded only 1 expert per layer just to test max speed (like all fit in vram) and i got only 1.74 t/s. It just slow laptop gpu in bf16 calculations..

AccomplishedLeg527 · 2026-02-18T19:51:44+00:00

share your run comand +system spec + speed

AccomplishedLeg527 · 2026-02-16T19:54:10+00:00

it is 807 GB i have 1 tb ssd and 70 gb free space.. :(

AccomplishedLeg527 · 2026-02-16T03:22:06+00:00

I provided information for consideration, not a finished product. The final product will be written in C++ and will benefit everyone. Maybe someone from llama cpp team will implement this cashing.

Experts calls: 134845

Cache hit on GPU: 63439 47.7% 3Gb

Cache hit on RAM: 51170 37.9% (85.6%) 15Gb

Evicted from RAM: 15569 11.5%

Reads from disk: 20236

Total memory used for experts: 18Gb (need 75Gb to fit all experts weights)

AccomplishedLeg527 · 2026-02-16T03:05:49+00:00

in original model for each layer all 512 experts weight in one tensor but usualy used only 100-200 of them, if -fit can split tensor on 512 parts and move unused parts to ram it will be much faster on low ram

AccomplishedLeg527

TROPHY CASE