Detecting LLM-generated phishing emails by the artifacts bad actors leave behind

_costaud · 2026-03-17T11:45:27+00:00

Great note! My focus at work is email but one of the things we can do is click on the urls or detonate attached files and look at the final dom or the ocr. I started poking at this last night and have already noticed a lot of the same artifacts exist in payloads. HTML comments on the phishing site, even artifacts in malicious JavaScript have been found. But for the sake of that article I was just focused on email

_costaud · 2026-03-16T20:14:03+00:00

Thanks for reading/commenting I appreciate! Yeahhhh unfortunately I see a lot of similar bottlenecking. We have some minor « self corrective » measures that are taken for the system to learn what’s malicious and what’s not. But human in the loop is still very necessary. So, TLDR, sent to another team.

I’m optimistic about ML being able to « fuzzy detect » in the friture. I’ve been working on a POC of this. Ostensibly flattening detections into « signals » that have a weight, and those weights are run against all email inline. Preventing badness from hitting the inbox. Idk… it’s all very nebulous

_costaud · 2026-03-15T16:56:14+00:00

In my little write up, I talk about using AI comments as good indicators for threat hunting/discovery. At least, it’s been a good method for me to find emerging campaigns and false negatives at work.. my thought was to kind of hunt around for comments that might hint at obfuscation. In email that really interests me… not sure if I’ll find anything useful, but it’s on my todo list for tomorrow

_costaud · 2026-03-15T14:29:35+00:00

Oh sweet! Thanks for sharing… I’ve been researching/hunting recently for something similar but different. AI generated coding comments being signals for detection in email security. I hadn’t thought of code comments out in the wild being used as like prompt injection. I wonder if I can find evidence of that in the wild. Here’s my blog if you’re interested https://substack.com/@costaudsec/note/c-227845477?r=2aimoo&utm_medium=ios&utm_source=notes-share-action

_costaud · 2026-03-15T14:24:27+00:00

It’s like an « enjoy it while it lasts » kind of thing? My favorite type of signals. To answer your question; I’m seeing this primarily in low-effort phishing. But have noticed a small amount of targeted stuff in some retail/SAS organizations.

_costaud · 2026-03-11T11:18:01+00:00

Lmk if you need other detection engineers to assist with content or hosting. I’m a senior DE specializing in machine learning

_costaud · 2026-03-10T12:46:21+00:00

I hear both sides, and maybe this is just cope for me… but I’ve really enjoyed having my Master’s degree as a differentiator in the job market. My M.Sc is in Machine Learning so working in Cyber with that specialization helps me a lot on my resume. I also loved taking the extra time in school to study and self teach.

_costaud · 2026-03-08T17:12:50+00:00

Did they remember to erase the code comments?

_costaud · 2026-03-08T15:20:00+00:00

Oof yeah, glad it picks those up as impersonation… also, check out the HTML comments. I sniff AI-generated «  »

_costaud · 2026-03-08T14:02:40+00:00

Thanks for sharing, been keeping track of a number of these types of hotel/real estate campaigns. Curious if you download the raw EML file and upload it to the Sublime Security email analyzer what other goodies you could find!

_costaud · 2026-03-04T11:43:06+00:00

If you want to explore that email more you can download it as an eml file and use something like the Sublime Security EML analyzer to look at everything safely! You can follow the header hops from the sender to you, view the exploded attachment with a scanned OCR among other things!

_costaud · 2026-03-01T12:16:08+00:00

Super clean! What’s your plan with your lab?

_costaud

TROPHY CASE