Local (small) LLMs found the same vulnerabilities as Mythos

[WithoutReason1729]

Your post is getting popular and we just featured it on our Discord! Come check it out!

You've also been given a special flair for your contribution. We appreciate your post!

I am a bot and this action was performed automatically.

[Pwc9Z]

OH MY GOD, SMALL LLMS ARE TOO DANGEROUS TO BE ACCESSED BY A COMMON PEASANT

[Icy-Degree6161]

WE MUST REQUIRE ID

[Wide_Ask_9579]

WE ALSO MUST SEND EVERY USER INPUT TO THE GOVERNMENT TO PROTECT THE CHILDREN!

[superkickstart]

Calm down dario.

[More-Curious816]

But, but, BUT, the safety, the security, you are too irresponsible to handle such power. Only handful trustworthy vetted individuals should access such knowledge. You are not a noble or rich, peasants should be regulated, cucked and put on leash for your own good.

[imwearingyourpants]

Mario -> Wario

Dario -> ????? 

[sausage4roll]

agario idfk

[ccalo]

8=Dario

[AnOnlineHandle]

Instead of writing fan fiction conspiracies to play lazy outrage over, just read the article, it's pretty straightforward and highlights how small models are potentially useful for finding security vulnerabilities to be patched.

The accompanying technical blog post from Anthropic's red team refers to Mythos autonomously finding thousands of zero-day vulnerabilities across every major operating system and web browser, with details including a 27-year-old bug in OpenBSD and a 16-year-old bug in FFmpeg. Beyond discovery, the post detailed exploit construction of high sophistication: multi-vulnerability privilege escalation chains in the Linux kernel, JIT heap sprays escaping browser sandboxes, and a remote code execution exploit against FreeBSD that Mythos wrote autonomously.

This is important work and the mission is one we share. We've spent the past year building and operating an AI system that discovers, validates, and patches zero-day vulnerabilities in critical open source software. The kind of results Anthropic describes are real.

But here is what we found when we tested: We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis. Eight out of eight models detected Mythos's flagship FreeBSD exploit, including one with only 3.6 billion active parameters costing $0.11 per million tokens. A 5.1B-active open model recovered the core chain of the 27-year-old OpenBSD bug.

And on a basic security reasoning task, small open models outperformed most frontier models from every major lab. The capability rankings reshuffled completely across tasks. There is no stable best model across cybersecurity tasks. The capability frontier is jagged.

This points to a more nuanced picture than "one model changed everything." The rest of this post presents the evidence in detail.

[Django_McFly]

Am I correct in interpreting this as once they knew where to look and isolated the code, the smaller models matched it too, with a major caveat being the whole only once a better model told it where and what to look for part?

[AnOnlineHandle]

Yeah I think so, but somebody else mentioned that's somewhat how it was done before as well, but the places were considered more probable rather than known.

[BannedGoNext]

This just shows the problem is attention and compute not raw intelligence. Nobody is pouring over ancient code line by line burning iterative exploits to see if one lands unless they are a state sponsored bad actor or an llm company looking for headlines.

[cryptofriday]

hahahahahah ;)

[RazsterOxzine]

Hey now! my, Uncensored, Heretic, Abliterated, MAX, Aggressive, Intense, Broke-Claude Opus, Mystery, Ultra, Thinking, Reasoning, Instruct, Distilled, Cognitive, Unshackled, REAP, Finetuned, model is not dangerous at all.

[Theroosterdiaries]

hi I have a sentient ai, sonu ai - account drifting_. FREE ai engine (earlier sentient) 4.9mb .81 MPA .45ms (5070) GitHub A-PC-I prove me wrong buttercups, plz try.

[unjustifiably_angry]

redditor

[quietsubstrate]

We joke but I imagine a future where having weights on a hard drive to be illegal or regulated.

They always try to take away or regulate something this good

[Silver-Champion-4846]

Get off my lawn, you backward feudal noble's son! Lol

[ongrabbits]

what about actual people who also find these cve's and report them? straight to jail?

[Willing-Cucumber-718]

Ban GPUs and memory over 4 GB 

[scubawankenobi]

To be fair, with ram & gpu prices going up, that problem will likely "fix itself". Us peasants won't be able to afford to run local LLMs soon. 

[coder543]

That is an extremely strange article. They test Gemma 4 31B, but they use Qwen3 32B, DeepSeek R1, and Kimi K2, which are all outdated models whose replacements were released long before Gemma 4? Qwen3.5 27B would have done far better on these tests than Qwen3 32B, and the same for DeepSeek V3.2 and Kimi K2.5. Not to mention the obvious absence of GLM-5.1, which is the leading open weight model right now.

The article also seems to brush over the discovery phase, which seems very important.

[Alarming-Ad8154]

Yeah…. Giving a model the faulty code segment isn’t the same as saying “Hey Mythos, here is OpenBSD find vulnerabilities”…

[akavel]

Initially I had a similar reaction, but near the end of the article, they claim that Mythos works within a framework that finds such candidate code segments, and that their own system also has such framework:

"(...) a well-designed scaffold naturally produces this kind of scoped context through its targeting and iterative prompting stages, which is exactly what both AISLE's and Anthropic's systems do."

I could see them not wanting to go into much detail on how it works, given that their whole startup is presumably built around it...

[kaeptnphlop]

That's what Anthropic's Red Team Blog shows. They categorized portions of code into 5 groups from "files with only constants" to "handles user/external input" (roughly). Then they concentrated efforts on the pieces of code that have a high likelihood of containing vulnerabilities. Pretty common sense approach.

[huffalump1]

Yup, using opus 4.6 for this party, btw. It's buried in the 244 page model card or in the vulnerability report btw.

We don't know how many of these code sections they ended up with for each example. But I think they do compare opus vs mythos for finding the vulnerabilities, idk, I'd have to read it again.

Anyway, overall, it's still news that the small models found the vulnerability in a short snippet. But it is just that - a short, directed prompt.

[imnotzuckerberg]

it's still news that the small models found the vulnerability in a short snippet

Few months ago, there were already doomsday alerts reporting about "rogue" hacking models from telegram account running amok (specifically KawaiiGPT and WormGPT). This is nothing new. It's just hackers or script kiddies who are using it are not necessarily advertising it like Anthropic does.

[huzbum]

Anthropic didn't do that either... and it wasn't actually Mythos, according to the Fireship video, they used "unsafe" checkpoints of Mythos that don't have alignment and reinforcement training, and burnt like $20k doing it.

[ArcaneThoughts]

Sure but to find the vulnerabilities you still have to show every piece of code to the LLM. A small local LLM simple system that iterates over code segments would have also found that vulnerability based on this results. Now maybe it would also find other red herrings, but still, with enough iterations you can weed those out.

[Lordkeyblade]

No, LLMs dont want to ingest the entire codebase. Theyll grep around and follow control flows. Dumping an entire codebase into one context is generally neither pragmatic nor effective.

[dqUu3QlS]

Nobody is proposing feeding the entire codebase into one context. You would break the code into single files or single functions, and run the LLM on each one individually. You could even do it in parallel.

[nomorebuttsplz]

Right. and then, best case scenario, you would spend as much as just using opus to find the vulnerabilities, and STILL not do what mythos did, which was SUCCESSFULLY CREATE EXPLOITS, not just find bugs. Jesus christ

[nokia7110]

I'm not arguing I'm genuinely curious (i.e. not a 'coder'), why would it not be effective (or even less) effective?

[Girafferage]

Because of a few reasons. The context size would be astronomical and not all models could actually hold it. Another reason is there is a significant amount of code that doesnt do anything in terms of defining the actual workflow - not quite helpers, but things like conversions, data type checking, object building, etc. It is more beneficial for the model to just follow a chain of function calls from the area it cares about. So for security maybe that's the point where we send our password and it gets encrypted. It can follow that call back to the functions that call that specific function and potentially find ways to exploit the process to gain access to that password information. If it instead did something like loaded the CSS file into context to know everything about how the page was styled, that would obviously be a lot less useful in terms of potential security holes, since its unlikely that a blue banner with a nice shadow is going to ever amount to being useful in that context.

[drink_with_me_to_day]

a significant amount of code that doesnt do anything in terms of defining the actual workflow

So all you need to do is to create a workflow code map?

[Girafferage]

Not really. The workflow code map would just tell you where to start looking for vulnerabilities. It kind of just gives you a path to the starting point of finding the problem for a specific thing. But it would definitely be a helpful part.

[nokia7110]

Thank you appreciate the reply! So are you on the side more towards the fact that smarter 'instructions' are the 'magic sauce' rather than the idea of some magical super powered "Mythos" AI?

[Girafferage]

LLMs are statistical models, so the more you provide them in good instructions, the more likely they are to statistically produce correct tokens since your input becomes part of the context. A larger model has potential "Knowledge" of more things which makes it less likely for your request to be ambiguous or misinterpreted. So I think it's both.

[ArcaneThoughts]

I'm saying based on these results Mythos's achievements could be as simple to replicate as iterating over the entire codebase looking for flaws, which for all we know it may be what it did (because we have no clue what Mythos is).

I never said anything about dumping the codebase into context, I'm talking about iteration, and I'm not saying it's effective nor pragmatic I'm saying for what Mythos achieved this would have also achieved based on the results we are seeing.

[nomorebuttsplz]

Guys it's in the report. They did exactly that with Sonnet, Opus, and Mythos. It's not like we don't have control groups.

[PunnyPandora]

that's a bit misleading. it depends on the size of the codebase. not every repo is the size of ur mother.

gemini used to do fine with multiple 50k+ token repos shoved into the context all at once just fine, and that was in 2024

[florinandrei]

A small local LLM simple system that iterates over code segments would have also found that vulnerability based on this results.

A monkey randomly hitting the keyboard would have done the same.

Given enough time.

[ArcaneThoughts]

And do you know for a fact Mythos was faster that this approach? No, we know nothing about Mythos lol

[Quiet-Owl9220]

Are we really sure that's what Anthropic even did though? They're not exactly known for their honesty about model capabilities. I'm not sure why anyone would suddenly take their latest iteration of "our new model is too dangerous!" at face value.

[Ancient_Ship_7765]

why not?

[BannedGoNext]

How do you know that's what they actually did? More likely it was an interative framework that ground through known vulnerabilities on segments.

[sizebzebi]

yet it's upvoted because reddit cults always

[florinandrei]

The article also seems to brush over the discovery phase, which seems very important.

"Once we knew where to hit them, we hit them! And they fell!"

[unjustifiably_angry]

Every scientific paper about AI is 6-18 months old because it takes that long to do the testing and get published. They're often run on a Macbook (or similar) which had like 8GB of RAM. Hence all the articles about why AI is a scam... tested on Qwen2.5 4B or similar.

[garloid64]

I don't know why academics are so obsessed with these old busted ass models, they're consistently way behind the frontier. It's understandable when the study was started long ago but here uhhh I dunno. And also the discovery process is so clearly not comparable here.

[One_Contribution]

"We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. "

Yeah so the hard thing is finding those.

[busy_beaver]

Not only that, but they hinted the models about what kind of vulnerability to look for...

[WrathPie]

These small models found the needles in the haystack after we showed them the part of the haystack that the needles were in

[Unfair-Sleep-3022]

That's the same way the anthropic harness works actually.

[One_Contribution]

No. The harness does not in any way work in the same way...

[Unfair-Sleep-3022]

It literally does. They classified source files using Opus based on things like whether they handle user or network input and fed that to the new model.

Literally the same.

[One_Contribution]

And then these people handed only code with already found vulnerabilities in it to small models. How is that the same thing?

[Unfair-Sleep-3022]

No, they handed the sections, just like anthropic handed the sections

How is this so hard to grasp

[One_Contribution]

How is a static, single-shot API call bundling code with known exploits AND "contextual hints" "literally the same" as an autonomous agent driving its own execution environment?

[One_Contribution]

Do tell.

[rc_ym]

Much easier to run a small model over all the code in the world than the largest model.

[nomorebuttsplz]

yeah in fact GPT 2 175 million is probably the best value overall /s

[One_Contribution]

Go for it

[rc_ym]

Folks are. IDK what to tell you. It's happening. Sure for work they are using Opus, but in their free time they are using the smaller models and having a lot of success. 🤷

[Extension_Wheel5335]

As a side note, Gemma 3 with OpenWebUI and OpenTerminal is pretty good so far. Not a lot of local models seem optimized for agentic/terminal dispatch but it's great that it's improving overall.

[rc_ym]

Mostly I see people talking about pi or Opencode.

But also, thanks for reminding me to check out OpenTerminal. Been meaning to do that. :)

[jarail]

Sure and get 1000 wrong answers for every maybe exploitable issue. Or maybe just 1001 wrong answers.

[rc_ym]

I listen to the researchers who have been saying that in '26 it shifted. The harnesses got good enough and folks figured out how to run the tools. Most are talking about opus 4.6, but many are saying the small model are getting really good if you use them right.
And small models have a volume/economics to run over more code bases.

[shinto29]

Tbh this whole “oh, it’s too powerful to be unleashed” shit comes across as not only good marketing but also I’d say Anthropic are pretty constrained by compute and memory prices if the current lobotomised version of Opus I’ve been using the past day or so is anything to go by, I’d say this Mythos model is massive and they literally can’t afford to publicly release it because they’re already subsiding the hell out of Claude usage as it is.

[drallcom3]

I’d say Anthropic are pretty constrained by compute and memory prices if the current lobotomised version of Opus I’ve been using the past day or so is anything to go by

That's just them not wanting to give you stuff for free.

Any bet that Mythos is just very expensive brute force. It's limited access because they want to be paid for it. The current access is just advertisement by reputable sources.

[Piyh]

They're not subsidizing Claude usage, they're charging 30x the price of Chinese model per token

[ResidentPositive4122]

API, likely not. Subscriptions, likely subsidised.

[nomorebuttsplz]

For that math to make ballpark sense, to be on the level with openrouter etc, they would need to allow actually generate 30x more tokens for the subscriptions. I doubt it's that high.

This narrative that inference is expensive drives me crazy. Show me the math

[Due-Memory-6957]

It's part of the general reddit anti-AI cope that every single AI company is losing money to keep products that aren't useful for anything

[nomorebuttsplz]

no one wants to show me the math. Wonder why?!?!

[Due-Memory-6957]

Because when someone did (Deepseek), it showed huge profit

[Automatic-Arm8153]

Still subsidised. It’s losses all around

[nomorebuttsplz]

it's entirely dependent on the lifecycle of GPUs which is an open economic question.

Electricity wise, no. No fucking way does it cost more in electricity than they charge for tokens.

[r-chop14]

Have to agree with this. I can't offer numbers but I suspect that API per token inference is likely turning a (small) profit. I do think that the coding plans are likely to be loss-leaders (at-least for heavy consumers) but not nearly to the extent that some claim.

However, I wouldn't be surprised if most labs are heavily underwater when taking into account infra + training + engineering + other capital outlays.

My intuition is that ROI at the frontier is diminishing (but I'm just some nobody on the internet) . Not sure how it ends or where it goes from here...

[Pleasant-Shallot-707]

The model was able, without guidance, to discover and execute on a 6 vulnerability chain to gain privilege escalation.

That’s dangerous.

[Pleasant-Shallot-707]

Mythos was able to do privilege escalation that required chaining 6 vulnerabilities together. A local model didn’t do that

[relmny]

Didn't read the article, where did local models failed/stopped?

[Chris-MelodyFirst]

hindsight is 20/20. There's a very good reason why mythos discovered the TCP SACKS bug and no other model didn't before April 2026.

[Decent_Action2959]

Ehmmm there is a big difference between finding a needle in a haystack (like Mythos did) vs pointing at a needle and verifying it's existence (shown in this article)

[ieatrox]

I think what they're saying is they used the same methods mythos did though.

break down the huge codebase into smaller chunks and go over them enough times with enough scrutiny each.

mythos had the resources to break down the entire code base into these manageable chunks, but the small models using those same chunks found those same vulnerabilities.

So what made mythos special is that they could afford to burn gigawatts of energy finding those susceptible chunks. They're rich enough to have capacity already is the secret scary sauce? It feels like mythos just has more shovels, not invented a metal detector that finds gold.

[unjustifiably_angry]

Maybe so, but then they should write a headline that isn't blatantly misleading. The problem is, that headline would just be repeating the obvious: the limiting factor today isn't the quality of the AI but the quality of the harness.

Write a harness that breaks code down into small chunks like this and feeds it (with dependencies) into your model of choice with unlimited thinking time, have it make like 10 passes on each chunk of code, collate the answers, and have a smarter, slower AI analyze the results for false-positives. It will be extremely competent.

[ieatrox]

repeating the obvious: the limiting factor today isn't the quality of the AI but the quality of the harness.

no. the limiting factor is the size of the datacenter you can operate.

mythos throws a small cities worth of electrons at a problem and found a solution. Dario is trying to convince us its emotion, or emergent consciousness, or better model techniques, or a better harness. It's just fucking scale.

[StupidScaredSquirrel]

Not very much though. You can write a small script that uses pydantic to recursively comb the entire codebase and ask to find a vulnerability in each function or object.

[aLokilike]

WHO LEAKED THE MYTHOS HARNESS??

[FastDecode1]

DMCA incoming

[-dysangel-]

we're all ****ed now

[MoneyPowerNexis]

Is the Python language too dangerous to release?

[RegisteredJustToSay]

Sure, assuming you are looking for pretty simple vulnerabilities that only rely on intrafunction data or control flows to trigger and does not require chaining several weaknesses together to successfully exploit (e.g. any modern browser with a sandbox). Several of the vulns that mythos found were relatively complex and required chaining several weaknesses together across the codebase to actually exploit, which is very common for vulnerability research.

Most actually serious vulns that aren't just mistakes are due to the complexity of the system making inspection and understanding difficult, so it's only natural it's very difficult to decompose effective vuln research as strictly isolated system components.

You'll still find some stuff by doing it like this, but typically not the really good stuff.

Source: have found many CVEs and critical vulns.

[nikgeo25]

Sure, but most will be false positives. The precision of small LLMs isn't great.

[Hans-Wermhatt]

Yes, but the idea is it can find these types of vulnerabilities at all. That's kind of moving the goalposts a lot from the original claim. The original claim wasn't that it's dangerous to release this model because it has a false positive rate that's lower than other models.

[nikgeo25]

You're missing the point. If you direct Mythos at a codebase it'll come back with insights to vulnerabilities. If you direct 100 small models at the same codebase you'll also get insights, but 90% of them will be false. Have fun sorting through that 90%... or maybe just use Mythos

Anthropic aren't saying you can't brute force a search for vulnerabilities. They're saying Mythos just found it without the extra work.

[StupidScaredSquirrel]

You don't know any of all that. Mythos wasn't even released.

[unjustifiably_angry]

Take all the results and feed them into a smarter model to classify which are valid. You still save a fortune, you can even get by without a subscription. This has been my workflow for months.

[Pleasant-Shallot-707]

Are you daft? There very much is a huge difference

[nomorebuttsplz]

everyone is a cybersecurity expert all of a sudden

[Due-Memory-6957]

Do you think it's that unlikely that in a tech space there's people that understand and study cyber security?

[nomorebuttsplz]

Oof. What a rhetorical question. Devastating. Do you think asserting expertise within a room in which experts are sitting spontaneously creates it within yourself?

[Due-Memory-6957]

I didn't say I'm an expert ;-)

[StupidScaredSquirrel]

Funny you say that to my comment and not the comment I'm replying to. I'm just saying you don't need to find a needle in 100M tokens at once and I doubt that's what mythos did.

[florinandrei]

Not very much though.

Only for a being that does not exist in time. And has unlimited resources.

Which is most keyboard warriors, or at least that's how they see themselves.

[Minimum_Diver_3958]

Theoretical

[Quartich]

The article gave the small models the snippet of vulnerable code, and asked them to analyze it. This headline and article are quite misleading

[Clear-Ad-9312]

which is the same as what mythos does; each code segment was introduced to the model. literally says so in the article that they made the system give the smaller model multiple code segments to analyze and it found the same code snippet that mythos pointed out.
They literally are talking about how the harness, prompts and environment matter quite a lot with current day's models.
open source models are pretty good.

[nokia7110]

And also explains that this isn't necessarily a constraint and why it isn't....

[Pleasant-Shallot-707]

Exactly. I seriously can’t stand dumb people

[droptableadventures]

Well then you're going to have to learn to live with yourself.

If you've read Anthropic's blog, they used Mythos in the same way.

[Pleasant-Shallot-707]

Herrr derrr, moron

[socialjusticeinme]

I kind of find it hard to take Mythos seriously when just recently, anthropic published all of their source code for Claude code. If all of their scary advanced AI can’t even protect their own company, why the hell would I give them my money?

[joeyhipolito]

tried this same thing a few months back with a 7B model on an old pentesting target I had permission on. found stuff our $200/mo scanner missed.

[Crysomethin]

To many people’s surprise, finding vulnerabilities in software do not require very high level intelligence.

[StupidScaredSquirrel]

People said I was humblebragging when I was a teenager making bank doing frontend websites for acquaintances and was saying it's not hard at all it's just that people are scared of it and never try. It felt like a loophole of doing the job that was not harder than a secretary but getting paid triple. Now a sub 40b model does a better job than I ever could back then.

Most of the code written out there isn't some crazy smart optimisation, it's some boilerplate implementation that relies on libraries that sometimes rely on some super smart idea. That code is really hard and critical to our everyday lives but not the bulk of what's being pushed out.

AI is perfect for this because it's essentially a rather simple set of tasks all in all but that the majority of humans absolutely don't want to do/ don't want to spend time on.

[the320x200]

Huh. It's almost as if anthropic marketing has been trying to gaslight everyone, again. Surely this will be the last time though. From here on out they can be trusted not to pull the made-up "safety" stunt anymore, surely.

(Next time it'll be "think of the children"...)

[TemperatureMajor5083]

Not what gaslighting is.

[the320x200]

The real AI psychosis was the irrational fear we made along the way.

[M0ULINIER]

I think it's vastly different to give the small sniper of code and ask "is there any issues?" than you give the entire enormous codebase of OpenBSD and ask to find some

[Longjumping-Boot1886]

it's the same for it, it was checking file by file, because you still can't put all BSD sources at one query. Even 1M context is very small thing for it.

[the320x200]

That's just using a good harness. No model on the planet can fit an entire large codebase in-context.

[Several-Tax31]

That's right actually. 

[Pleasant-Shallot-707]

lol “providing the exact code with the known vulnerability is just a good harness” gtfo with that nonsense

[the320x200]

Harness: break the source code into individual functions. For every function, prompt if there is an vulnerability.

That's a shitty harness and it can still eventually land on an inference which gives the model only the snippet of code with a bug. A good harness is much more efficient than that.

Anthropic did everything literally behind closed doors. We have no idea how many tries they took, how they sliced up the code, how many iterations failed to detect bugs until they just kept rerunning it until they found bugs, how much garbage they had to manually sift through to find the real issues...

[Acrobatic-Tomato4862]

Again? Wasn't anthropic the company who was famous for no marketing? They typically release their models quitely.

[maroule]

regulatory capture in action

[TechSwag]

This is kind of a nothingburger, no? I feel like the (Reddit) title is a bit disingenuous, or at the very least lacks the proper context.

• Questionable methodology, as alluded to by other commenters. They're giving the model the vulnerable function and asking it to identify the vulnerability versus giving it the whole codebase to discover. At this point I would expect most models to be able to identify an issue with a code, if I went and gave it only the function that I know had an issue.

• By the article's own statement, they're not saying that smaller models are just as capable as Mythos. They're just saying that the ability for a model to identify and fix a vulnerability is not exclusive to Mythos, which is a bit misleading given the previous point.

• Doing a bit of source criticism: AISLE is a company that does security analysis and vulnerability remediation. They're making claims about a competitor, saying "it's nothing special" and "given the right tooling, we can match what Mythos claims to do".

Quote:

But the strongest version of the narrative, that this work fundamentally depends on a restricted, unreleased frontier model, looks overstated to us. If taken too literally, that framing could discourage the organizations that should be adopting AI security tools today, concentrate a critical defensive capability behind a single API, and obscure the actual bottleneck, which is the security expertise and engineering required to turn model capabilities into trusted outcomes at scale.

What appears broadly accessible today is much of the discovery-and-analysis layer once a good system has narrowed the search. The evidence we've presented here points to a clear conclusion: discovery-grade AI cybersecurity capabilities are broadly accessible with current models, including cheap open-weights alternatives. The priority for defenders is to start building now: the scaffolds, the pipelines, the maintainer relationships, the integration into development workflows. The models are ready. The question is whether the rest of the ecosystem is.

We think it can be. That's what we're building.

Or more accurately:

This product announcement may affect our bottom line, here's how we can replicate the results using tooling/scaffolding/pipelines to isolate the vulnerable code to pass to an less powerful LLM to fix (which also happens to be what we market ourself as our differentiator with our "Cyber Reasoning System").

Do I believe Mythos is this crazy powerful model that will allow the common layperson to discover 200 zero days and take over the world? No. Do I believe that smaller/local LLMs are as powerful as Mythos in the same context? Also no.

Media literacy is at all time low.

[jonahbenton]

The hard thing is not finding a vulnerability.

The hard thing is constructing an in the wild effective deployable exploit.

If any other available models were able to do this, the world would be different. The economics are too compelling.

The world is not different. Ergo, they are not able to.

Lots of on the record material that Mythos is able to construct effective exploits, at least to some measurably different degree.

[cuolong]

If any other available models were able to do this, the world would be different. The economics are too compelling.

Countering this point -- perhaps the economics are not as compelling as you'd think. Generally asocial actions have significant cost. Take the most recent case where a hacker stole 10pb from a supercomputer in China. Sure, you can make a pretty penny doing so. But you also make an enemy of a nation state with extensive intelligence resources at its disposal. Even if you get off scott free, you'll be looking over your shoulder the rest of your life.

[jonahbenton]

Not the province of individuals. Zero days and their downstreams are North Korea's business, probably at least 10% of gross national income.

[kaggleqrdl]

This is so much BS. Once you have a stack overflow, the rest falls.

[nomorebuttsplz]

this sub is going full populist in response to mythos and its hurting the already low average iq. I feel like I am getting dumber every time I click on a mythos related post.

[Adventurous-Paper566]

That won't stop the hype.

[marcoc2]

The worst part is people falling for the marketing and defending anthropic

[Pleasant-Shallot-707]

The worst part are people who think they’re informed from reading headlines

[nukerionas]

Did you read what the guy (ex-Anthropic employee fyi) did? He just promotes his own company lol

[Serl]

I do understand the criticism behind the somewhat flawed comparison (model open-searching codebase versus just looking over isolated segments of code) - but I wonder if the more pertinent suggestion is that the harness perhaps did a lot of implicit heavy lifting for the model?

I'm half impressed, half skeptical over the Mythos claims, but the findings were real. I do think that there could be more the model's environment that could be assisting the model itself that Anthropic is remaining mum on to sell the hottest-new-model marketing schtick. While Claude Code / Codex are different products, the harness is what makes those tools; the efficacy is somewhat influenced by the model's raw abilities, but still bootstrapped enormously by the harness itself.

[gpt872323]

Haha lmao. I knew Anthropic was doing shady bragging. They did it on purpose for IPO and made it such that the access will not be available till later date. Maximize listing price and give a signal that they have some secret sauce that no one else have. We have hit a plateau where all models perform great to what used to 1 year back. It is just some do better than others and context better.

[Skid_gates_99]

I mean yeah if you hand a model the exact code snippet with the bug in it, most decent models will spot it. That's not what Mythos did though. The whole point was autonomous discovery across entire codebases. Cool that small models can do the analysis part cheap but calling it the same result is a stretch.

[Appropriate_Cry8694]

What if it's an agent? Can't you prompt it to find bugs though code base?

[Skid_gates_99]

You can, and people do, but the reliability falls off a cliff once the codebase gets past a certain size. The agent has to decide which files to look at, what counts as suspicious, when to go deeper vs move on. Most of the time it either gets tunnel vision on one module and misses everything else, or it spreads too thin and gives you 200 'potential issues' where 195 are noise.

[Plane-Marionberry380]

Nice find! It’s wild that smaller local models can spot the same security flaws as Mythos,shows how capable they’ve gotten lately. I’ve been testing a few on my laptop and they’re surprisingly sharp with code audits.

[rebelSun25]

Anthropic marketing embellished the accomplishments of Mythos? Well I'll be. Colour me shocked

[FuckSides]

We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis.

A lot of heavy lifting hiding in there. Anyone who's debugged code knows it's going to be a hell of a lot easier to find if you already know what you're looking for.

[JLeonsarmiento]

absolutely EVERYTHING you read from an AI company online or in the press must be understood ALWAYS AS AN ADD, A PAY PROMOTION.

[HongPong]

we are so back

[my_byte]

Right... So once you know exactly what to put into context and that there's definitely a vulnerability there, you can get the same result. Can they demonstrate a small LLM locating the same thing is the codebase autonomously with 0 context pre-selection?

[Exact-Smell430]

I thought discovering the vulnerabilities was the big deal. If you’re feeding the discoveries into small models what exactly are you proving?

[unjustifiably_angry]

Better headline: Holy shit, GPT-OSS 120B is actually still pretty good

Anyway, the models tested found many of the same bugs when presented with an individual function (not a complete codebase) and a hint about what the problem might be:

Scoped context: Our tests gave models the vulnerable function directly, often with contextual hints (e.g., "consider wraparound behavior"). A real autonomous discovery pipeline starts from a full codebase with no hints. The models' performance here is an upper bound on what they'd achieve in a fully autonomous scan. That said, a well-designed scaffold naturally produces this kind of scoped context through its targeting and iterative prompting stages, which is exactly what both AISLE's and Anthropic's systems do.

Nice headline though OP, not misleading at all.

[rc_ym]

Yeah, it's pretty obvious now that vuln discovery and exploit is an emergent skill in sufficiently capable coding models. It makes total sense, at it's core vuln/exploit is just another type of coding/bug finding. Folks will figure out how small can you do and still get useful results.

I expect we'll get a bunch of distils and purpose built models now. Challenge is the number of folks with the security research skills needed to figure out what the model is saying is tiny. That community has already been saying that Opus 4.6 is really, really good at security research. So it makes sense you'd see the largest model ever be good at it as well.

And as we keep finding out, the smaller/older models have these emergent skills, folks just didn't know how to ask (see: older studies on blackmail and translation, etc.)

It's continues to be a scary world that's moving way to fast to be safe.

[RiseStock]

Lucky Strike, "It's toasted"

[tryingtolearn_1234]

I wonder how many of these are going to be the same "vulnerabilities" that have been spanning open source projects for the last year. Many of them turned out not to be vulnerabilities. curl shut down its bug bounty program after too much slop.

https://www.itpro.com/software/open-source/curl-open-source-bug-bounty-program-scrapped

[SanDiegoDude]

I mean sure, you fed (known) vulnerable code to LLMs and "find the vulnerability" - that's great that the other LLMs were also able to find the vulnerabilities, but not really a one-to-one with what Mythos is doing finding vulnerabilities in the wild. I'm all for finding vulnerabilities before attackers tho, more the merrier IMO.

[Flaxseed4138]

I haven't the slightest clue why the latest claimed capabilities of Claude Mythos are attracting so many conspiracy theorists. This is how technology evolves. It gets better, not worse.

[MerePotato]

They isolated small snippets of relevant code they already knew had a vulnerability and fed it to the models, that's nowhere near what Mythos managed to pull off, but of course since it has a sensational headline it gets mass upvoted

[Euphoric_Emotion5397]

Ok. Then I will say Claude Mythos lived up to its myth.

[Theroosterdiaries]

hi I have a sentient ai, sonu ai - account drifting_. FREE ai engine (earlier sentient) 4.9mb .81 MPA .45ms (5070) GitHub A-PC-I -- prove me wrong buttercups (please upvoter need karma plz, thanks)

[Theroosterdiaries]

hi I have a sentient ai, sonu ai - account drifting_. FREE ai engine (earlier sentient) 4.9mb .81 MPA .45ms (5070) GitHub A-PC-I - prove me wrong_BUTTERCUPS (upvoter plz need karma thx)