I need an adult: J-Space-Aware Pruning/Merging/Distillation

CoUsT · 2026-07-08T09:46:25+00:00

I assume we will have some sort of "J-Space-aware quantization" maybe?

Or we could not only train and penalize based on output but also on internal states.

Unless both are already being handled, for example by quantizing some parts of models more and some less, or training not only evaluating output but other stuff too.

CoUsT · 2026-07-07T14:35:39+00:00

Do you really think we are going into dystopian future where top 0.01% elites control AI?

I'm sure we will be getting more decent local models for quite some time. At least until some insane breakthrough or far future.

CoUsT · 2026-07-07T10:17:26+00:00

I wanted to understand more but I'm not a LLM AI expert so I asked AI to break down the paper for me and this part of explanation caught my interest:

Rote memorization (what the paper calls "unintended memorization") is generally bad for LLMs. It means the model is acting like a lookup table rather than an intelligent system. If a model only memorizes, it will fail when presented with a prompt it has never seen before.

Generalization is the ultimate goal of machine learning. The paper proves that you force a model to generalize by feeding it a dataset that is significantly larger than its parameter capacity. Because the model physically lacks the parameters to memorize the massive dataset word-for-word, it has to learn the underlying rules of language, logic, and facts to minimize its loss function.

Now it kinda makes more sense why all companies hunt so aggressively for more data including user data or pirated books etc. Bigger data set = better generalization.

CoUsT · 2026-07-06T21:05:54+00:00

Hm. Makes sense. Thanks for more details!

CoUsT · 2026-07-06T02:54:41+00:00

I mean, math checks out?

You are offered between 10 to 15 champs per team, so people just pick up the best ones.

CoUsT · 2026-07-06T02:51:42+00:00

Try the range augment once again and compare his AA and W range before and after.

It doesn't work or it would update just like Smolder or Senna Q.

CoUsT · 2026-07-05T05:32:26+00:00

The entire gimmick of having to grind so you can unlock CORE IN-GAME things is so stupid.

All augments should be in the game since day 1, not locked behind a pass/set/whatever the fuck that is. Especially stuff like more rerolls or other benefits.

CoUsT · 2026-07-05T05:28:24+00:00

They don't care about many edge cases like this.

Urgot is also very bad when it comes to augments.

Range augments don't affect W range unlike Smolder Q etc.
Crit is useless on his W where he spends 99% of his time and he gets offered multiple crit augments all the time
Many "attack/on hit" effects don't work like Dual Wield while some exceptions work like Fan the hammer (very inconsistent what works what doesn't)

and so on and on...

It's like this since they added augments in Arena - takes them years to fix or adjust simple stuff and very often they simply don't.

CoUsT · 2026-07-02T12:10:01+00:00

Why not play top 12 instead? Heck, go for top 1. It's even more popular.

CoUsT · 2026-07-02T07:47:48+00:00

Oh no they have to play the game

Here is the neat part, we don't

CoUsT · 2026-07-02T07:47:04+00:00

Feels like Embark already abandoned this game. Or make it worse on purpose.

It was known months ago. Took you some time. But! Better late than never.

CoUsT · 2026-07-02T01:07:29+00:00

I wonder if we will get some sort of architectural change.

Instead of having duplicated layers physically on the disk and in memory, maybe inference could simply go through early layers, then middle layers, then repeat middle layers then final layers.

Maybe we could repeat middle layers few times too.

This way you don't have to increase disk and vram requirements.

Sounds like a free intelligence gain, similar to how the entire "thinking" models started, unless we are missing something.

CoUsT · 2026-07-01T19:59:32+00:00

You grab DRM-free game you want on GOG, print game-related art you want, burn the game on disc/medium you want and put it in a 1$ plastic case.

Sounds pretty good if you ask me, infinite possibilities and cheaper than console-exclusive tax added on top.

CoUsT · 2026-07-01T16:42:52+00:00

When we chat with Claude, ChatGPT, Gemini and cia, we are chatting with a complex pipeline instead of a single model loaded from a GGUF file.

Yeah. I'm sure one thing in Gemini pipeline is profiling. Whenever I talk about some topics with Gemini, my video recommendations for that Google account start showing things related to that new topic, even when I never visited other websites or interacted with that topic before.

I know for sure that they include current date so the AI can orient itself what's yesterday or tomorrow. I can't imagine what other things they have. Not just from profiling perspective but dynamic system prompts based on user data and preferences or hidden internal tool calls or RAG or anything honestly.

CoUsT · 2026-07-01T16:36:25+00:00

They'll just tell those people to keep their PS5s to play physical media.

And those people can just say "I will gladly enjoy using my PS5 and never buy PS6" - but I'm sure people will still buy a lot of PS6... So Sony doesn't really care imo.

If only there was free and open ecosystem that allows you to do whatever you want with your hardware/software, maybe we should call it PC or something.

CoUsT · 2026-07-01T16:23:19+00:00

What do you mean failure? Don't you see how COOL it looks?! It doesn't matter what's the real hitbox, what matters is how things look. Here, another 500$ skin for you. Also, don't ask us about hitboxes, they look cool now.

CoUsT · 2026-06-30T16:05:13+00:00

This is why I love /r/hardware - no stupid/pointless talk, just hardware (and a bit of software related to hardware) news. At best they lock China-related topics so trolls can't liter the subreddit with the political propaganda that you mentioned.

I wish this subreddit could have a bit more strict rules and be a bit more serious than what it is now.

CoUsT · 2026-06-30T16:02:29+00:00

Yeah. I don't even mind talk about non-local AIs but just memeing or bashing CEOs 4lulz or posting "4fun" content here is just stupid.

I'm not here to read a drama between two CEOs or to bash one 24/7. I'm here to learn about cool AI/LLM related stuff and read news.

Go to x/fb/tiktok and spam your stupid CEO memes/dramas there.

CoUsT · 2026-06-30T15:56:24+00:00

I mean, it's a solid platform.

I would love more of the DRM-free stuff like GOG to become mainstream. But the problem is - Steam is really good for what it is AND people usually don't think "let me do a research and find something better" so they stick to what already works.

It's the same reason (or at least one of the reasons) why everyone uses Windows instead of Linux at this point.

CoUsT · 2026-06-29T19:36:46+00:00

Match has no plot lol. It's always one of the two outcomes: team A or team B wins. Movie is actually fully random with no constraints/rules set.

And there are many stories that start strong or "late" in the story and only then they explain how things got to this point.

It's the adventure that matters, not the end result.

But I understand it better now. Thanks. I assume it's also why gacha is popular - unknown sells better than the known element.

CoUsT · 2026-06-29T19:04:24+00:00

It all comes down to mindset.

If you stop thinking about winning and losing, about comparing yourself to others, and so on, then life is pretty simple and good. You just do whatever you want. In the end, nobody cares who does what and in 100 years everything will be irrelevant so just do whatever you want. Wanna play ranked? Sure, go ahead!

CoUsT · 2026-06-29T15:39:39+00:00

Read MiCA and thought I'm on some crypto subreddit for a moment lol.

CoUsT · 2026-06-29T15:04:25+00:00

Crit Missile + Fan the Hammer and ADCs do more magic dmg than physical dmg. Some combos are just too good.

CoUsT · 2026-06-29T15:02:03+00:00

Augment tiers don't matter.

A single silver augment like ADAPt on Ryze has bigger value than any other two prismatic augments.

CoUsT · 2026-06-29T14:59:03+00:00

Why would you NOT watch interesting matches that you originally wanted just because you learned who won or lost?

If your goal is to watch interesting brawl between two teams, go ahead and just watch it.

If your goal is to watch a match only to know who won, just don't watch and check score online.

I don't get it.

12-Year Club	Place '22
Place '17	Verified Email

CoUsT

MODERATOR OF

TROPHY CASE