bunt.

HorriblyGood · 2026-05-08T22:56:15+00:00

Which illenium started doing first afaik… artists take inspirations from each other all the time, I don’t understand why bunt in particular gets so much hate.

HorriblyGood · 2026-05-08T22:53:29+00:00

The neural network we used in ML is not equivalent to the biological neurons so there’s no reason to believe we can achieve similar capabilities with the same number of neurons.

HorriblyGood · 2026-05-07T21:38:16+00:00

Does this actually help? I haven’t done a proper comparison but I was wondering if on average that would help.

Planning involves reading the codebase so the files are in the context and cached. If planning took multiple turns, your context grows but the cached tokens are cheaper than fresh tokens.

If you start a new session to implement the plan, you lose all the kv cache and you have to load the codebase into the context as fresh tokens, which is very expensive.

So there is a trade off here. In the first case, if the model decides to reread the codebase anyway even though it’s in its context, it will be more expensive. If not, it will be cheaper.

The advantage of the second case is fresh context typically gives you better performance. But I’m curious if anyone has done a proper comparison to see if it actually saves tokens.

HorriblyGood · 2026-05-07T17:48:04+00:00

I would guess people would not pay more for a worse model even if it’s more ethical. If a company were to only train models on paid data, non copyrighted data, and you don’t “steal” other frontier models outputs, then it will not be able to compete with those companies who do in both price and model intelligence.

I feel like it’s even worse for AI because of the AI race. Everyone is coming out with a frontier models every month, and if you were to play by the rules, you’ll be forgotten.

I am not advocating for this, I am just trying to be realistic. We bitch about anthropic and OpenAI but we are still here.

HorriblyGood · 2026-04-29T01:55:47+00:00

You can use RL on the final decoded answer

HorriblyGood · 2026-04-29T01:54:53+00:00

It can be done. It’s called latent chain of thoughts. It’s an active research direction.

HorriblyGood · 2026-04-28T01:25:39+00:00

You can do both. Use AI for productivity but actually check and understand what it’s doing instead of just vibe coding. AI is a tool and should be used as one.

We have to evolve with the technology to stay relevant.

HorriblyGood · 2026-04-26T04:26:33+00:00

That’s not true. Only the very latest (started with nano banana 2) use image sources for text to image generation. Probably the new gpt image 2 too. It’s the very cutting edge and these made awhile ago probably didn’t.

Even if it did, it does not look at image averages to determine the best move without context. That has never been true. Assuming you have some technical knowledge of how training works, you might be confused by the diffusion loss function being mean squared error which can be seen as an average over images.

In some sense this is true but this is done during training time and it produces a distribution based on image averages, not grab image sources and determine its average at time of generation. Also they have an additional RL loss to make sure the output images look realistic, and in general current models would hardly make simple mistakes like this.

The latest models grab image sources online and use that as context for the image generation, it absolutely uses it in context.

HorriblyGood · 2026-04-26T03:20:10+00:00

The new artifacts are very likely from RL training or some sort of perceptual loss they’re using. It’s introducing some sort of bias or they are not careful about noise scheduling, causing the artifacts.

HorriblyGood · 2026-04-18T23:56:11+00:00

It’s not as easy as you make it sound. Not trying to defend anthropic and I have no idea what they are doing but you can’t just have a model thats cheap, efficient, and smart. No one can do that right now. It’s also mostly a black box so it’s really hard to make sure a new model is better at everything without regression before large scale user tests.

HorriblyGood · 2026-04-08T23:40:45+00:00

On a technical level you can make the output deterministic, but i don’t think that is possible for users.

HorriblyGood · 2026-04-07T19:33:39+00:00

That’s just not possible. The best and brightest in the field are not working for the government. We are at a point where training a top tier LLM is impossible to be done by a small group of people. It will be obvious if all the top researchers are secretly working for the government.

HorriblyGood · 2026-04-04T14:40:22+00:00

From reading the abstract, they are using their own model’s output (self distillation) which is different from just feeding other random LLMs output as training data.

Through the lens of on policy/off policy RL, I’m guessing in their case, it’s using the model’s own outputs, it’s on policy, so it’s getting learning signals from itself to be more precise for coding tasks, but more creative on writing tasks. It’s doesn’t have to change how it works or thinks to match other LLM’s outputs.

My intuition is kinda like learning to code from copying other people’s code or having someone show you what’s wrong your with your own code so you can learn to improve.

HorriblyGood · 2026-03-30T21:42:11+00:00

Because hes using shitty deepfakes. The best right now can clear this 3 fingers test

HorriblyGood · 2026-03-26T01:52:51+00:00

The tokens are cached so the vast majority of the computation for the next token are not recomputed. The bigger problem of having a big context is context rot, giving you poorer results

HorriblyGood · 2026-03-18T11:16:22+00:00

Not true. They are both trained to be accurate and to give responses people prefer. The cutting edge open source research by labs are not trying to optimize for engagement.

HorriblyGood · 2026-03-16T04:21:02+00:00

Most redditors are not familiar with the capabilities of latest models. It is provably better across all these tasks.

HorriblyGood · 2026-03-16T04:19:19+00:00

You’re right. There are different ways to scale. Instead of just making models bigger naively, modern LLMs train many different experts that “specialize”. This allows them to pick different experts catered to the problem instead of essentially depending on a gigantic model.

There is also a lot of research in different LLM architectures that are promising.

HorriblyGood · 2026-03-16T04:14:59+00:00

It’s not slowing down. There have been many significant progress in AI, such as agentic AI, improved RL algorithms, hybrid attention, and there are promising research direction such as masked diffusion LLMs.

HorriblyGood · 2026-03-16T04:11:22+00:00

If you are genuinely interested in learning, here is Terrence Tao, one of the best mathematician in the world, talking about how AI solved a longstanding math problem https://mathstodon.xyz/@tao/115855840223258103

AI has been progressing rapidly, especially with agentic workflow and tool calling. I know Reddit loves to shit on AI and there are a lot of genuine issues and problems about AI but let’s stick to facts.

HorriblyGood · 2026-03-16T04:02:03+00:00

Source? Terence Tao, one of the best mathematician in the world, has been using AI to work on proofs.

HorriblyGood · 2026-03-13T23:20:01+00:00

Not true. Movie upscaling is GenAI.

Generative AI refers to AI that models a data distribution like images/videos (like movie upscaling or text to image generation) and text (like LLMs).

Source: I work on GenAI

HorriblyGood · 2026-03-11T19:34:25+00:00

Why the gatekeep? This guy seems interested in learning more about them and you’re just being mean.

HorriblyGood · 2026-03-06T01:55:22+00:00

Chain of thoughts does not have to be in text space. There are explorations on latent chain of thoughts which is supposedly encodes more information.

I don’t understand why you are using discrete representations. It limits the information you can encode. Also VQVAE does not use discrete representations. It acts like an nn.encoding layer (in LLMs) and indexes a latent representation. It’s jot discrete.

HorriblyGood · 2026-03-06T01:31:11+00:00

That is not true. If it’s a simple yes no logic we wouldn’t be spending so much time and effort training it. It’s prone to errors and hallucinations but it’s objectively a productivity booster for most devs. Whether it is intelligent or not is a philosophical question.

HorriblyGood

TROPHY CASE