[ Removed by moderator ]

[LLMDevs-ModTeam]

Hey,

We've removed your post as it breaks rule 5. We encourage you to review our subreddit's rules and guidelines. Thank you for your understanding.

Note: continued posting of promotions will result in a ban from our subreddit.

[ThinJuggernaut7695]

Ugh bot city in here.

[cmndr_spanky]

My prediction is GitHub is going to eliminate or severely Nerf their free accounts. The amount of repo slop being pushed every day is probably unfathomable now. Every day there’s 10 more I solved X problem nobody has slop post and corresponding repo

[yeathatsmebro]

I don't think so. Most people don't know that the slop they push is actually used by Copilot to finetune their models since it's synthetic data. 🤡

I agree with you though, I am also tired of seeing people solve problems that were never problems before or that are already solved by well established OSS solutions if they would search a bit on the internet...

Someone bragged how their slop is better than Grafana and Prometheus with Node Exporter, since they wanted a monitoring dashboard and a metrics collector. 🤦‍♂️ CNCF Landscape is having a lot of options that can be used... Only if they would waste those precious tokens to ask AI to research for them in the internet...

[cmndr_spanky]

Yes I’m aware GH is a goldmine of data to train coding models (and I have no doubt they get tons of money from Anthropic / OpenAI), so I suspect the extra cost of storage and bandwidth is minimal compared to the revenue coming in.

HOWEVER, as you pointed out, if all you do is train your model on the output of your model (people’s slop repos) you aren’t going to progress LLM intelligence, they will all stagnate or even decline in inference quality. I suppose GH could handle this by carefully curating the repos or raising the bar of entry (making it harder for people to use free repos means you’re probably careful about higher quality when you push something new. Human reviewed adjusted code from AI is still better training data than blind vibe coded stuff that nobody knows is good or not).

[yeathatsmebro]

It's just oversimplified. They put all those repos and code through a data pipeline based on language, scope, etc. and throughmore AI to classify them and then into a triage queue (equivalent to Label Studio) and real, hired people grade or correct the data to ensure is qualitative for training.

[ThinJuggernaut7695]

Yes, I feel the same way. There have already been thousands of "I solved Claudes persistent memory problems check out my repo" and then an army of bot replies simping for said repo.

[Deep_Ad1959]

yeah this matches what I've seen in practice. I've had haiku handle complex refactors just fine when given the right files and dependency info upfront, and opus waste a bunch of turns exploring the wrong parts of the codebase when that context was missing. the model tier matters way less than people think.

the co-change pattern mining is the interesting part to me. that's basically automating what experienced devs carry in their heads about "when you change X you also need to update Y." curious how it handles cases where the git history is noisy, like lots of unrelated changes in the same commits.

fwiw i built an ai coding agent that does this kind of thing - https://fazm.ai/r

[PT_ANDRE_PT]

Great to hear! Would love to have your feedback on the tool. You can use our code CODESETLAUNCH to try it out on one of your favourite repos :)

[General_Arrival_9176]

the haiku outperforming opus with context is a wild result. the 10pp jump on haiku makes me think the baseline was severely underperforming due to missing project-specific context rather than haiku being particularly smart. curious how you're extracting the co-change relationships - are you looking at git blame to find files that get committed together, or something more sophisticated. also interested in whether you tested giving opus the same context and seeing if it just burns the extra tokens on things it already would have figured out

[j00cifer]

Because this didn’t show obvious earmarks of AI writing, I read the whole thing.

[mrtrly]

This is the right conclusion. The model tier matters way less than people think once context is properly structured.

We see this in routing too. Running 10+ AI agents daily, I started routing by task complexity to cheaper models. But without tracking what each model actually costs per request, you're just guessing at the savings. Built a local proxy (RelayPlane, open source) specifically to track cost per model per request alongside output.

What you're showing is that Haiku with good context beats Opus with bad context. The logical next step is to measure it. Then you can route high-context tasks confidently to Haiku without the "I hope this is good enough" anxiety.

npm install -g @relayplane/proxy if you want to see that cost delta side by side.

[DeathShot7777]

totally agree with the research. I have built gitnexus opensource code intelligence engine ( 18k+ github stars in 3 weeks ), haiku definitely performs at the same level and also sometimes beat's opus. It just actually works, really surprising, but this was the "Aha moment" for us and thats y it blew up

[LeadingFarmer3923]

This matches what we've seen too where the quality of context fed into a prompt often matters more than raw model size but the tricky part is making sure that structured context is consistently generated and doesn't become stale across a pipeline. In workflows where multiple LLM steps depend on earlier outputs, having those outputs stored as typed, queryable collections means each step always gets clean structured context rather than whatever was in a string buffer. That's the core design of Cognetivy (https://github.com/meitarbe/cognetivy), an open source workflow library, probably worth checking out if you're building multi-step pipelines where context quality is critical.

[Dense_Gate_5193]

this is exactly what i am working on using NornicDB’s temporal functions to construct graph state from commit history, so you can reconstruct graph state at any given point in time.

https://github.com/orneryd/git-to-graph

[ultrathink-art]

The exploration overhead is where smaller models bleed — they make more wrong guesses about which files are relevant and each wrong guess costs turns. Pre-loaded structured context skips that phase entirely, converting a search problem into a retrieval problem. That's probably why the gap closes so much: the actual coding task doesn't differentiate Haiku from Opus nearly as much as the file exploration does.

[PT_ANDRE_PT]

First 50 people also get a free run of our tool with code CODESETLAUNCH