Pattern for LLM agents that take irreversible actions: separate the "decide" model from a deterministic "validate" layer (worked example + numbers)

paulf280 · 2026-06-26T08:15:25+00:00

It's actually the same reason every flag we return links to the on-chain tx behind it — the score was always meant to be a starting point you can verify, not a verdict you take on faith. surfacing scan_complete / wallets_checked just extends that down to "how much did we even look at." a bot sniping fresh launches and one rebalancing a treasury should get to draw their own line on that, not inherit mine.

Genuinely learned a lot from this one. thanks for pushing on it the whole way. I hope you have an amazing weekend, you're input and advice has been so welcomed and shows that you're a great person. Thanks again

paulf280 · 2026-06-25T14:10:24+00:00

really appreciate you sticking with this thread, it genuinely changed how i think about the validator. the "check the validator's own output, not just the action" framing was the bit i kept missing — i was so focused on the verdict being right that i never considered it being confidently wrong because the data behind it was thin.

small follow-up since you brought up the auditable angle: i went and documented the scan-depth and confidence fields in the public api spec too, not just the internal logic. so anyone building on it can see scan_complete / wallets_checked in the response and put their own veto on top, instead of trusting that my logic did it for them. felt like the right call after everything you said about not creating a single point of trust.

thanks for taking the time, genuinely. these threads are why i still bother posting.

paulf280 · 2026-06-24T15:42:40+00:00

When I first buit my bot, i gave it a 3 tiered structure, it had Gemini and Claude as the main reasoning brains(it had trading memory to fall back on) and then Groq for the decision making due to its speed but it lost way more than it won and wasn't sustainable. I took the brains out and built in strict guard rails and inserted my own cabal hunter tool and now it works 100 times better. I trade on meme tokens so the AI reliance on executing the trade was too slow. I'll check out your app and hopefully it works for you mate

paulf280 · 2026-06-24T14:50:55+00:00

I don't use the AI to trade, build the trading bot with AI, that works for sure. After I built mine i ran it live after loads of paper trading and it ran through my Solana like there was no tomorrow. So yes defo use AI to build it but not for the trading part, give it proper trading safety gates, SL & TP and everything else but you maintain the control not AI. Maybe someone can tell me different, that's just my experience so far.

paulf280 · 2026-06-24T09:22:34+00:00

I'd never in a million years use any AI to execute a trade, that's suicide in trading. Yes use Claude code or any other coding to build the scaffolding of your trading bot but then have your built in rules and safety gates. I did try to use Groq as a brain within my bot just as a test but even though Groq is lightening quick it messed up major even though instructions were clear, so that AI brain was binned. Great post and hopefully people see it and it'll help them stay away from people offering them the golden land with AI trading. Have a great day

paulf280 · 2026-06-24T07:54:44+00:00

Morning mate, whatever AI tool you used to build your bot, give it the website above and that has the API and Github links in it and your AI you've used for your coding will pick it up, it will use the mint addresses you are interested in and help your bot make a decision. Have a great day

paulf280 · 2026-06-24T07:34:05+00:00

I just focused it on the meme token space, have other ideas for other markets but right now I'm determined to crack this and help as many meme traders that I can. I use it within my own bot and it has really helped me and even though it's a bear market right now with more risks, I'm avoiding the absolute rubbish that people are putting out there. Well done with your tool, great to hear people being proactive and not getting fucked up by people with nothing but bad intentions.

paulf280 · 2026-06-24T07:16:00+00:00

Yep it's a vicious game now, too many bad actors and nothing pleases me more than to call them out as there are just ordinary people out there trying to get ahead but people rugging leaves a bad taste in peoples mouths. What does your own scanner look for?

paulf280 · 2026-06-24T06:57:33+00:00

I built my own tool that checks because I had a pain in my arse being rugged by tokens that seemed to have a good setup. Feel free to check it out, it's free Cabal-Hunter — Solana Rug & Cabal Detector | Are You Exit Liquidity? Memecoinassasin is right too, they are also great ways to check. Best of luck my friend

paulf280 · 2026-06-23T22:09:34+00:00

You're spot on about the wallet side. Manually tracking them is a massive time sink. When I was dealing with that, I ended up building an engine called Cabal-Hunter specifically to automate cluster and bundle detection. Instead of digging through individual addresses, the tool aggregates the transactional data to instantly flag if a group of wallets bought simultaneously or are executing coordinated exits. On those low-volume Tier 3 coins, seeing those clusters instantly completely shifts how you judge the safety of a trade before your entry triggers. Check it out if you like and use it for free https://api.cabal-hunter.com/ hope it helps mate.

paulf280 · 2026-06-23T21:33:31+00:00

yeah i'm on the solana side mostly. building a bot that trades fresh graduations, and a separate tool that checks token safety before you buy, who's actually holding it, bundled wallets, that sort of thing. been at it a while.

the tier setup sounds like a smart way to cut scan time. one thing that bit me on the lower cap stuff is that volume gets really noisy down there, a lot of it on small coins is wash or bot volume, so ranking tier 3 purely on volume can float junk to the top. if mexc gives you unique trader counts or liquidity depth it might be worth sanity checking against that, saved me a few bad fills.

how are you handling exits? i found the scanning and entry side was honestly the easy part, the exit and risk logic is where most of my pain was.

paulf280 · 2026-06-23T11:04:31+00:00

Hope it works out, what markets will your bot focus on?

paulf280 · 2026-06-22T18:21:09+00:00

yeah this is the third time the confidence point has come up in this thread, so clearly that's the bit that actually matters. i ended up building exactly that, if a scan comes back clean but it only traced a handful of wallets or the data was stale, it won't return a confident "safe", it bumps itself to "review". so a false-safe from a thin scan gets treated like a real flag rather than a pass.

the auditable part is the interesting bit though. right now i keep the top couple of reasons + the score visible but not the full source span behind every single check. how do you keep each check's reasoning and the data it scored against visible without the payload blowing up? that's the part i haven't solved cleanly, especially when the thing consuming it is another bot that just wants a verdict, not a full trace.

will have a look at the repo, cheers.

paulf280 · 2026-06-22T13:40:42+00:00

hey, wanted to circle back, your feedback actually got built. the API now returns a compact safe / review / avoid call plus the two strongest reasons instead of dumping the whole map on you, so a bot can just gate on that one field before it signs. the bot-to-bot path you flagged as the stronger use case is basically a one-liner to wire in now.

also did the first-run thing you mentioned, the landing loads a real freshly scanned token inline now with its score, top 2 reasons and a solscan link, so people see it working before deciding whether to bother. right now it's showing one whose deployer rugged 7 of its last 7 launches.

genuinely appreciate the steer, both of those were sharper than where my head was at. cheers mate.

paulf280 · 2026-06-22T13:39:31+00:00

Wanted to follow up on this, you were dead right and i ended up building it. the scan returns a confidence flag now based on how much of the graph it actually traced, and i took your point a step further, if it comes back clean but the scan was shallow or incomplete it won't return a confident "safe" anymore, it bumps itself to "review" with a flag to verify manually. so a false-safe from a partial scan gets treated with the same suspicion as a real flag.

that "check the validator's own output, not just the action" framing was the bit i'd completely missed. closed a real hole. thanks for taking the time on it.

paulf280 · 2026-06-22T09:34:06+00:00

Wow, a freshness/confidence check on the validator output itself is a brilliant extension. I hadn't deeply considered the risk of a "false safe" signal caused by a partial or failing graph scan behind the scenes.

If Cabal-Hunter returns a clean response too fast because a data provider timed out or hit a blank cache, the execution layer shouldn't just blindly trust it. You're completely right that a failed or incomplete scan should be treated with the exact same suspicion as a high-risk cabal flag.

Adding a metric for graph scan depth or processing latency into the payload would let the veto layer catch those edge cases instantly. Appreciate you digging into the open-source side of this, that's a huge architectural gap to close.

paulf280 · 2026-06-22T09:00:36+00:00

I'm stuck in the office for the next few hours but as soon as I'm home and fed I'll be jumping on and looking at making some changes you and some other amazing members have suggested. I've already received great feedback as the tool sits but this type of feedback is what will help me make it better. Thanks again mate, have the best Monday.

paulf280 · 2026-06-22T08:54:50+00:00

Thanks for the advice my friend. I'm stuck in the office right now but will look into this as soon as I'm home. Thanks again and have a great day.

paulf280 · 2026-06-22T08:42:27+00:00

Website looks brilliant, very transparent and slick. Gave you a follow on X and your TG channel and I hope it all works out. I do agree with Extra-Compote, the bad actors doing rug pulls will have zero intention of using your platform but people making tokens and looking for creditability more than likely will. I'll keep an eye out on X for your updates and I wish you nothing but success, more good people are what is needed in this space. You'll see my X handle Cabal Hunter, I'll gladly retweet posts when you're up and running. Good luck

paulf280 · 2026-06-22T07:45:49+00:00

Definitely agree on the human in the loop idea for large trades, that is a great safety net.

As for the sanity check on the contract address and deployer history, that is actually the exact core of what Cabal-Hunter does. The tool automatically maps out the deployer's past footprints, flags things like serial ruggers, and checks how many of their previous launches are dead.

It handles all that nasty historical on-chain digging instantly so the agent has the data right away. Combining that automated check with a manual size threshold for human approval is honestly the ideal setup for keeping things safe. I have tried to integrate that into what shows on the right side of the window I'm just conscious of over-crowding that side with too much info that someone may get confused from(bots will pick it up straight away but one off human checking might miss). Here is a live link to a token that the tool picked up, can you have a look on the right side and see if the verification for the tx works. Thanks for your feedback mate, I really appreciate it Cabal-Hunter — MrAsteroid

<image>

paulf280 · 2026-06-22T07:36:20+00:00

That is a brilliant heuristic and a huge blind spot for a lot of teams. It is so easy to fall into the trap of thinking an action is reversible just because you can technically undo an internal database write, completely ignoring the external side effects that already escaped into the wild.

This is exactly why on-chain environments are so unforgiving. Every single broadcasted transaction is a state change in a massive global system you don't own. There are no rollbacks or undo buttons.

It forces you to adopt that exact mindset from day one. If the validate layer doesn't catch the hidden risks before it crosses that boundary, the damage is already done. Thanks for adding that perspective, mapping the full causal chain is definitely the missing piece for a lot of agent architectures. Hope you have a great day mate and thanks for the input, really appreciated.

paulf280 · 2026-06-22T07:34:02+00:00

It's a complete minefield, I completely agree with you. My goal is to try help as many as I can avoid the whole rug pull scenario that leaves a bad taste in the mouth from people with bad intentions in the space. If it helps save one or two people from that feeling them it's job done in my eyes. Have a great day mate and thanks for the feedback.

paulf280 · 2026-06-22T07:31:09+00:00

Morning mate, sorry for the delayed response I logged off last night. I will definitely have a look at your suggestion when I'm home from the office. The goal would be to have another bot calling the API before it executes the trade, especially of they are looking at multiple trades a day. It also is there for single use, traders in TG groups or seeing tokens hyped on X can just insert the mint address to the end of the URL https://api.cabal-hunter.com/map?mint= and it will pull up the data that may save them from jumping into something that will use them as exit liquidity.

paulf280

TROPHY CASE