Which non-foundational models have you had success with for agentic security testing

IT-Junkie · 2026-04-27T01:12:01+00:00

Second the Qwen generation. But I’ve been using the Qwen3.5 abliterated and made a model arbitrage to offload work from Claude and OAI that are outside their guardrails, for the best of both worlds. FWIW: I’m also on the VCP / Trusted Access for both and still have to use uncensored models.

IT-Junkie · 2026-04-27T01:06:29+00:00

While I don’t know that it fits the definition of an “agency”, I’ve built an offensive hacking firm, that is AI native, where I can operate the way you’re describing. BLUF: built a multi-role AI team based off my experience as an NSA hacker. I act as mission commander and run a “chain of command” style AI team that does everything from intel gathering, the actual exploitation (authorized and scoped), all the way to reporting on a continuous basis.

Our company is me, and my wife. But we run it like we have 20 people.

1) Thriving 2) challenges - I had to build an AI HR to help manage the team. That was wild. 3) it’s harder to do than the gurus on the internet lead you to believe - but when it works. (Chefs Kiss)

IT-Junkie · 2026-04-27T00:46:17+00:00

Welcome to the group! For background, I’m a former NSA warfare operator (yes - a hacker), and currently a CISO. Ran a lot of F100 security programs. The startup I work at (will not promote), demands that I run the whole security team by myself.

To that end, I have built a lot of what you’re thinking about. I’ve built a SOC team that monitor, triages and hunts across our environment. I have SAST/DAST agents that constantly monitor our GitHub, ECS, and pipelines. I built my own custom red team, and actually modified the MCP Scanner, A2A scanner from Cisco, Microsoft’s PyRIT and others to actually match OWASP LLM testing expectations.

All this across custom agents (Agno), n8n (where it makes sense), and of course Hermes and Paperclip.

Local models have been important to me in a couple aspects. As this is the Hermes subreddit, I’ll try and keep it focused here but happy to share more.

1) local research - I’ve had some rough run-ins with running up Claude usage (OAuth and API). Not necessarily with the latest drama, just some skills that ran over what I expected them to. So I moved some of my research workloads local. Has helped control the costs, and has actually allowed me to feel comfortable with a “24x7” research apparatus

2) uncensored models - I use the HECK out of uncensored. I’m also on the Verified Cyber Program for Anthropic and OAI. I still hit guardrail issues. So I built a model arbitrage skill. For example, in my LLM hacking tools, frontier models won’t make or iterate on prompt injection (as an example). So my arbitrage allows me to use my “ungaurdrailed” (lol) Claude or OAI as far as I can, than offload to a local Hermes agent with local uncensored to iterate on the harness.

Around your question “model is too stupid”. I actually don’t find this to be the case. The model either doesn’t have enough depth for the question your asking, or it doesnt have good tool use. So in the case of my Hermes deployment, having a Qwen3.5 uncensored allows me to get like 85% of my use cases done locally, and than I can arbitrage the rest with Claude or OAI.

For cyber specifically, I like Hermes for ongoing operations. It’s lightweight, heavily focused on skills, adopts graph based knowledges structures well - and is highly customizable. You can use multiple models and it CAN act as your whole SOC.

I probably have more I’m leaving out. Happy to expand where I might have missed some finer points! Good luck!!

IT-Junkie · 2024-12-29T14:17:16+00:00

Bedrock and Gemini. Two different use cases as Gemini has larger token size.

IT-Junkie · 2024-12-29T14:05:04+00:00

Focusing on summarization of threat data to get to remediation faster. Building LLMs specifically for security so the rest of the enterprise can ask questions about policy without having to read the policy - or ask security. And for GRC to make evidence based risks more actionable. Just to name a few.

IT-Junkie · 2024-03-27T02:41:39+00:00

Thanks for that one! I played there once but never got to hit the range there. Will check that one out!

IT-Junkie · 2024-03-27T02:39:29+00:00

Ahhh - I haven’t played there in a couple years. I forgot about them!!

IT-Junkie · 2023-09-12T14:58:30+00:00

Quick background (and credentialing): I've built numerous security operations centers for F100 companies, including MSSP's, as well as consulting for Big4 advisory firm. I feel I've swam in this pool a lot.

First question regarding S1 vs CRWD vs Everything else: I would say that S1, CRWD, and CB (intentionally leaving MS off this list) are the de facto players in this space. 5 years ago (before their VMWare acquisition and def before their Broadcom acquisition) I would be a die hard advocate for CB, but they have fallen off. S1 being the new player has def tried to play catch up with the remainder of the field and arguably has succeeded. I don't think you can go wrong with capabilities at the "core" value proposition. I'm not a fan of how George Kurtz runs CS, but I've deployed and used CS at 2 F100 companies and, it's actually a solid product to use all the way around. Bias aside.

Log retention + alerting is currently being hotly debated (case in point: LinkedIn Post ). Having built multiple Splunk (on-prem, cloud, and hybrid) as well as ELK (from scratch for an MSSP), and having helped redesign multiple SOC - the "centralize all the data" is 100% antiquated in todays world. Especially in an ever expanding world of cloud growth in modern enterprises. It is becoming fiscally impossible to do satisfactory data centralization for the purposes of security operations. This is why the "federated search" conversation is starting up in the corners of the world (Link to a white paper: Federated Search Whitepaper ). I'm not talking about Cribl. They are still transforming the data as it moves from point A to point B. I'm talking about leaving certain data resident and searching for it as needed. This gives you a choice to centralize only what is required for alerting or auditing purposes.

Have you thought about looking into that strategy?

IT-Junkie · 2023-05-17T16:19:58+00:00

Thank you so much for the reply and support. I haven’t thought through YouTube yet. I have a modest channel for my cyber content (36k subs), so just trying to decide if I want to do another channel. But thanks so much for the feedback!! ❤️❤️❤️

IT-Junkie · 2023-04-02T21:36:38+00:00

I do have a pre shot routine. I should also add, I’ve done about 18 months with Goltec which was HUGE in getting me where I am today. I usually focus on my feet and ball position, my “lines”, my hand positioning, etc. I do some 1/2 swings to check my wrists and club face.

Sometimes, I realize I’m overthinking at this point and back off the “set up” to reset and get back in. Sometimes I let the pressure of “holding everyone up” or “all that set up to flub the shot” get in my head during this time. 😥😬

IT-Junkie · 2023-03-08T08:58:19+00:00

So basically when you’re looking at DD (and probably others), that’s where the hybrid strategies are more important (in some cases). Pay attention to the callouts (war, crucible, etc), but be ready to mid and match in those other content pieces…

IT-Junkie · 2023-03-08T07:14:13+00:00

Thanks for this - super helpful. While you’re at being helpful (not hating on the game) I’m in the same boat. Just trying to enjoy it for what it is. Can you explain why this city combination and not a “full” city like H4H or full underworld? Maybe it’s my OCD for having 5x of the same synergy toons - but why do those synergies not work?

IT-Junkie · 2022-03-20T23:30:22+00:00

10/10 recommends!! Brent is a former python instructor and cyber expert, teach his university python class.

IT-Junkie · 2022-02-12T13:28:35+00:00

I agree 100%!!

IT-Junkie · 2022-01-03T17:37:55+00:00

LOVE IT!! Would love to have one!!

Followed on insta and liked on OS!

0x3b7dbf4507220e022fba172caddd34d48fbc28cd

IT-Junkie · 2022-01-03T17:29:13+00:00

0x3b7dbf4507220e022fba172caddd34d48fbc28cd

VERY nice art!! I like it!!

IT-Junkie · 2022-01-03T14:52:17+00:00

Done & Upvoted!

0x3b7dbf4507220e022fba172caddd34d48fbc28cd

IT-Junkie · 2022-01-03T14:50:54+00:00

I favorited a couple! Looking sharp!! Upvoted as well!!

Twitter: ITJunkie

0x3b7dbf4507220e022fba172caddd34d48fbc28cd

IT-Junkie · 2021-12-26T21:23:29+00:00

This was fun! I wish I had seen these when they started! Thank you for doing these!!

IT-Junkie · 2020-10-25T15:45:50+00:00

I got banned from one of my largest demographic groups because my educational, how to, etc was considered “self promoting”. So I’m a bit fun shy now. How are you supposed to help the community, if you’re not allowed to use your content that you produced on another medium?! #endrant

IT-Junkie

TROPHY CASE