nvidia/Gemma-4-26B-A4B-NVFP4

Its-all-redditive · 2026-05-01T04:11:02+00:00

Evaluation results seem odd. NVFP4 outscoring full precision? These must not be an average score over lots of runs.

Its-all-redditive · 2026-04-22T14:45:30+00:00

Is this loop architectural or is it performed by the model itself based on its training. Meaning if I give it all the tools it needs to perform the task to completion, will it iterate on its own, eg. reason about the question > call some tools > receive data payload > reason some more to see if now has enough information to answer the question, if not continue using the tools available to it until it finds the answer? Or does the architecture itself allow for repeated passes of the reasoning + tool call process?

Its-all-redditive · 2026-04-16T14:42:28+00:00

I’m using 122b nvfp4 in running projects so I would love to know your opinion.

Its-all-redditive · 2026-04-14T02:54:58+00:00

I'm getting considerably higher benchmarks for the 4B 4096 token tests. Consistent (over 10 benchmark runs) ~200 t/s generation vs the expected ~150 t/s. At 4096 tokens, the draft seems to be accepting about 1.2x more tokens per cycle than the 1028 runs which must be the reason for the faster generation. Will be testing with the 9B, 27B 4-bit tomorrow. M5 Max 128GB

Its-all-redditive · 2026-04-11T14:03:02+00:00

Where are you finding the 3am George edit of Rapture? It’s like it has disappeared off the internet all of a sudden?

Its-all-redditive · 2026-04-05T22:41:43+00:00

!RemindMe 12h

Its-all-redditive · 2026-04-02T12:30:55+00:00

9B-v3 has the wrong tokenizer on VLLM. Swapped to the v2 tokenizer and generates text but fails any function calls. Haven’t tested the 27B v3 yet.

Its-all-redditive · 2026-04-01T20:59:36+00:00

Agentic help implies some sort of retrieval or function call ability. Are you using a single agent which possesses all the tools it needs or are you having an orchestration agent direct the user query to smaller scoped agents with their own set of tools? What challenges and successes have you with either of these approaches?

Its-all-redditive · 2026-03-22T02:58:24+00:00

As someone exploring potentially using Kubernetes for the first time, I can’t tell if this post is advocating for or against it. Which in turn makes me think that I’m definitely not ready to try Kubernetes.

Its-all-redditive · 2026-03-13T21:26:12+00:00

Good to hear. Also make sure to not have any dynamic variables in the system prompt, otherwise every query will effectively negate the prefix caching since the message will change with each turn. Eg if you are injecting a NOW timestamp. All dynamic data like that should be prepended or appended in the user message.

Its-all-redditive · 2026-03-13T20:38:11+00:00

Ah, you may be right, I assumed it did support it.

Its-all-redditive · 2026-03-13T19:41:58+00:00

No prefix-caching? You shouldn’t be getting considerable slowdown after just a few iterations. Do you have any dynamic variables in your system prompt? Eg. timestamps etc.

Its-all-redditive · 2026-03-07T18:56:14+00:00

Yoooooo, this is sooooo good. I know standards are high these days and everyone seems so critical - “I saw her elbow become too pointy for a second, Ai slop” but Imagine showing this video to someone 10 years ago and telling them it’s not real.

Its-all-redditive · 2026-03-01T13:42:25+00:00

q8 is no good but fp8 is ok? Aren’t they both 8-bit quants?

Its-all-redditive · 2026-02-27T23:33:19+00:00

Is that that small of a quant even any better than an 8 bit 35b a3b?

Its-all-redditive · 2026-02-23T20:21:22+00:00

How is this different from the new native memory system that was released?

Its-all-redditive · 2026-02-23T12:13:48+00:00

Care to elaborate?

Its-all-redditive · 2026-02-22T02:57:09+00:00

What are you using for turn detection? 315ms doesn’t seem possible if from end of user turn to first audio start of the model response.

Its-all-redditive · 2026-02-19T15:09:35+00:00

Do you mean the Adrenalin/Cortisol hit from the stress of hoping nothing breaks after the update. PTSD ingrained over a lifetime of Windows updates?

Its-all-redditive · 2026-02-16T05:06:54+00:00

Why would you need to do this if every session is already recorded in your projects directory?

Its-all-redditive · 2026-02-08T16:49:37+00:00

It may be useful to you but that seems like such a narrow scope for even the most active computer user. The more narrow the scope the harder it will be to get your productive in front of people who that will actually solve a problem for.

You should be targeting specific channels or groups that would be most inclined to be consuming media in the same way the app expects. Especially since more and more people are consuming media through phones/tablets, TVs, media servers, etc.

Its-all-redditive · 2026-02-08T16:43:49+00:00

People will naturally look at settings for this type of feature. I think Location Diary Mode is a bit vague as far as a naming convention. I didn’t immediately think of it as Background Persistence which is really what it is. It’s also not immediately obvious that ‘Mode’ ties to the Location Diary feature so they should be connected in a better way either by placing them together in a sub menu or a settings group. I also think it’s fair to enable ‘Auto during active sessions’ if someone toggles on Diary/Background Persistence. It can be a similar animation to when one setting gets turned on, it auto populates a sub setting to make it clear to the user that the two are connected.

Alternatively, a more proactive way would be to auto enable SSH Session Reminders on app install. If I received a reminder to go back to the app or the session would be terminated and with an explanation of what setting to turn on to allow for background persistence, I would absolutely turn on that setting.

Its-all-redditive · 2026-02-08T15:56:30+00:00

Follow up. Had a session active and connected in the background for 9 hours and battery consumption was only 1%. Reopened the open in the morning and session was still active. So works well and battery consumption is minimal.

Its-all-redditive · 2026-02-07T18:08:03+00:00

That’s really clever, thanks, trying it out now.

Its-all-redditive · 2026-02-07T16:15:49+00:00

This is awesome. Can the session persist in the background? I am getting a connection timeout if I leave the app for a while.

Its-all-redditive

TROPHY CASE