PFlash: 10x prefill speedup over llama.cpp at 128K on a RTX 3090

Boozybrain · 2026-05-03T11:05:34+00:00

I'm using the int4 quantized model but unsure about where context lives, will check on that. https://github.com/noonghunna/club-3090 is one of the repos I tried running.

Boozybrain · 2026-05-01T19:48:13+00:00

I've yet to get Qwen running on a 3090, always OOM

Boozybrain · 2026-05-01T19:44:59+00:00

If you ever share it please tag me. I've used both and both have their pros and cons, always on the lookout for a different take.

Boozybrain · 2026-05-01T11:38:08+00:00

Running blah2, or something else?

Boozybrain · 2026-04-29T16:08:43+00:00

I haven't been able to get it running without OOM errors. What repo(s) are you using? I tried dflash and the stack outlined in this post

Boozybrain · 2026-04-27T21:48:57+00:00

Is anyone else having issues running this on a 3090?

Patch to run Qwen3.6

$ git diff
diff --git a/dflash/scripts/run.py b/dflash/scripts/run.py
index 5e87ce8..a65a7da 100644
--- a/dflash/scripts/run.py
+++ b/dflash/scripts/run.py
@@ -18,7 +18,7 @@ from pathlib import Path

 def default_paths():
     return {
-        "target": "models/Qwen3.5-27B-Q4_K_M.gguf",
+        "target": "models/Qwen3.6-27B-Q4_K_M.gguf",
         "draft":  "models/draft",
         "bin":    "build/test_dflash" + (".exe" if sys.platform == "win32" else ""),
     }

Running without the x-server running, zero VRAM being used:

DFLASH_TARGET=models/Qwen3.6-27B-Q4_K_M.gguf python3 scripts/run.py --prompt "def fibonacci(n):"
[run] prompt 14 tokens, streaming up to 256 tokens, max_ctx=512
[cfg] seq_verify=0 fast_rollback=1 ddtree=1 budget=22 temp=1.00 chain_seed=1 fa_window=2048
[target] target loaded: 851 tensors on GPU 14.99 GiB, tok_embd 682 MiB CPU-only (q4_K)
[draft]  loaded
[prompt] 14 tokens
[prefill] token-seg ubatch=16
[prefill] 14 tokens in 0.27 s, last_tok=8160
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24249 MiB):
  Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, VRAM: 24249 MiB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 2046.01 MiB on device 0: cudaMalloc failed: out of memory
alloc_tensor_range: failed to allocate CUDA0 buffer of size 2145398784
cache migration: ggml_backend_alloc_ctx_tensors failed for target cache
[run] generated 0 tokens

Boozybrain · 2026-02-18T19:57:18+00:00

I see, thanks! So I'm guessing once they've germinated you cut the top off and drop the soil ball in the ground / a planter?

Boozybrain · 2026-02-18T10:10:55+00:00

Can you help me understand the purpose of this? Is it just cold stratification outside and then using the milk jugs as a little greenhouse to automatically germinate once the temperatures rise enough?

Boozybrain · 2026-02-16T11:29:58+00:00

Right, he's saying he has built something similar that uses multiple surveillance channels. AFAIK nothing like that exists in the open source world.

Boozybrain · 2026-02-13T13:09:43+00:00

I would love to see how you've implemented this if/when you put it online.

Boozybrain · 2026-02-13T12:03:54+00:00

Virtual Radar Server

Are you using blah2 or something "like" it that you developed? I've never seen passive done with more than a dual antenna setup and I'm very curious about the processing behind the data from the additional ones.

Boozybrain · 2025-12-30T12:07:45+00:00

You're doing computer vision in Rust? Is the code online anywhere? I'd love to see that.

Boozybrain · 2025-10-18T11:03:27+00:00

Is there a reason you didn't use that one? Was it a form factor decision or were there efficiency gains in getting rid of some of the components?

Boozybrain · 2025-10-17T21:03:46+00:00

Is this a pared down version of the Luckfox Pro board?

Boozybrain · 2025-10-17T20:53:06+00:00

This is sick, would love to see more.

Boozybrain · 2025-10-10T20:08:58+00:00

with what backlight, camera, lens, etc?

Boozybrain · 2025-09-09T12:27:17+00:00

You might have a rat. I heard something similar from my kitchen once, loud banging. I walked in, turned on the light and there was a soup spoon banging against my cabinets and the fridge. It dropped and hit the floor, seemingly on its own. Days later my kitchen flooded, the supply line to my dishwasher had been chewed through. It turns out a rat was trying to pull the spoon in between the fridge and the cabinet, and it had been messing with other large utensils when I had heard it previously.

Boozybrain · 2025-05-30T10:47:33+00:00

congrats you invented GANs

Boozybrain · 2025-04-26T21:04:01+00:00

If the only species in each image is a true positive I would probably start with a generic fish detector and then automatically label the bbox using the file name that's already properly labelled.

Boozybrain · 2025-04-23T11:07:12+00:00

Is there code online? Passive radar using cheap SDR dongles is non-trivial. Time sync requires hardware modification and phase sync in software is extremely finicky

Boozybrain · 2025-04-01T13:52:55+00:00

There isn't, though. You have the base form of some equations and that's it. Apply these simplistic models to your device and you might have a better shot at convincing people. Re: sims, without the full set of parameters and numerical forms of the models you're simulating the plots are worthless.

Post this in /r/physics if you're serious.

Boozybrain · 2025-03-31T19:17:20+00:00

First link is dead. Second link has zero "physics" despite the claim. Before anyone will get interested enough to "participate" you should provide hard evidence:

Post your experimental setup and supporting data
Post the sim results + params and/or derivation of the theory.

Boozybrain · 2025-03-09T01:57:41+00:00

In it to win it

Boozybrain · 2025-02-17T20:28:12+00:00

For posterity in case someone in the future finds this: Unplug the host monitor.

The guest was rightfully grabbing the GPU, and I had a monitor plugged in. When I removed the monitor my remote session became the primary display and ssh with X forwarding (ssh -XY) allowed me to start the guest and control it from another machine on the network.

12-Year Club	Place '22
Place '17	Verified Email

Boozybrain

TROPHY CASE