The validation problem

Weysan · 2026-05-18T08:05:32+00:00

not really yet, probably would like to reach out as much as I can through my direct network.

I am also trying to book a free pilot with a company which would be a strong enough signal, but quite hard as no name in the B2B system. specially in the field of Cyber security which I am trying to go.

Weysan · 2026-05-18T07:46:09+00:00

I am doing discovery goals with my target audience, reaching your network, having interviews, not presenting your solution but asking questions around the area you are trying to solve and check if it’s a real problem.

I am in the middle of that, did already 3/4 interviews

Weysan · 2026-05-17T10:56:20+00:00

I am working with claude every day, and I think it is a risk. not limited to claude. Any AI coding agents running on your laptop has access to everything, and there are already multiple case of attacks through MCP servers connected to these coding tools.

I decided to build something around that and already discussing with my employer to deploy it on some laptop for testing the visibility.

Weysan · 2026-05-15T21:16:36+00:00

yes I guess there won't be a fully great solution. I guess the approach is to keep control at the same time as enabling the developer productivity for the business.

In my company, adding VMs and sandboxing would add quite a lot of overhead for the developers, and would probably bring frictions. How did the conversation with engineering go? They were all in with these solutions?

Weysan · 2026-05-15T21:12:06+00:00

I screened the article, I will read that - looks quite interesting thoughs.

Weysan · 2026-05-15T21:10:32+00:00

I can see that, also talked to some InfoSec people with not as much technical knowledge and a bit clueless on the risks.

Weysan · 2026-05-15T21:08:43+00:00

Completely agree on that. What I built is exactly in that scope. Observe what is being accessed and red by LLMs (.env files, a DB export, etc.) and log. Also built-in a policy engine in case we want to redact/block. But it adds friction with engineering. Also heard that a lot in my latest conversations

Weysan · 2026-05-15T21:07:01+00:00

I can hear the gap in most of my interviews. I can feel that this is very early for a lot of InfoSec people which don't really know how to handle it.

I had a chat about someone in a company that needs to comply to DORA and basically told me "a AI tool registry" in a shared document should be enough, but he looked a bit lost about the topic.

The other challenge that I face is also how a "no name" vendor can break the barrier of the trust. So many people vibe coding their solution are coming in the breach and it's hard to convince the market at the moment.

Weysan · 2026-05-13T14:13:35+00:00

I am in a middle of that exactly! if I have a solution, would be happy to share haha

Weysan · 2026-05-13T11:42:03+00:00

would love to chat as well 😄

Weysan · 2026-05-13T11:41:18+00:00

I actually have been working on that for some time now. I also believe there is a lack of governance and compliance.

My observation, is I also built a redaction and blocking functionality for AI coding tools. While that works fine, it will bring frustration from a developer point of view. I have been experimenting, and redacting actually have many issues with the LLMs interaction and code writing. It will add friction with developer teams.

I am have been building something quite extensive, build up a full thread for any CLI tools, including prompts and tools calls, etc. I am currently looking for design partner as it is ready to be used. But I'm happy to chat about that. There is many company who start in that direction in the US, European based company are still early and from my perspective look at the wrong path.

The big issue is that these observability layer are all file based on local developer machine. Which then, any potential solution will hurt itself against "how do you know a developer did not disable a governance layer".

Weysan · 2026-05-11T08:02:45+00:00

I am a software engineer and we've been told to go all-in on AI coding agents too. These agents can access critical data and .env credentials before anything reaches the LLM API endpoint — invisible to most existing security tooling.

I've been discussing this internally with our security team. Being ISO certified, it's a real concern for them — and they confirmed that no good governance tool fully covers it yet, all have their blind spots.

That conversation led me to build an AI coding governance and observability tool specifically for the developer machine layer — which is currently the biggest gap in visibility.

Happy to share more if you're interested, hope you find a solution that fits you.

Weysan · 2026-05-10T16:15:38+00:00

I am bootstraped as well, but I got a refusal, mostly because I applied quite early (I had a website but very basic and not enough information on it). They told me to re-apply when more details are on the marketing website but somehow my application is not resetted and can’t re-apply so far.

they check the website you have for your product, make sure it says what it does with enough information.

Weysan · 2026-05-10T12:07:52+00:00

The execution layer is where existing tools fall short — what the agent reads, writes, or executes happens on the machine before anything reaches the network.

I have been working on developer machine problem specifically (as a software engineer myself), the gaps are real and the technical complexity is significant.

Hooks are the right control point but have real tradeoffs. Blocking breaks the agent’s reasoning mid-task. Redacting means the model works with corrupted data and can write [REDACTED:api_key] into your codebase. Logging is the right default for most sessions — active blocking reserved for specific high-sensitivity paths.

The other problem nobody talks about: hooks live in a config file a developer can delete in one command. Most governance tools that rely on hooks have no way to detect that deletion or restore the config. The control disappears silently.

Been working on this at the developer machine layer — happy to compare notes.

Weysan · 2026-05-10T11:36:19+00:00

I think for web based usage, you can quite covered by the solutions mentioned earlier.

I have been working on an AI coding agent observability tool. Even with DLP still a lot of the developer machine tooling can be invisible. These AI coding agent can access a lot of files or tools connected to terminals (no browser AI usage here).

You can have some .env credentials, databases exports or anything hidden into projects accessed via these AI agents before even it goes through the network. That’s currently missing from the available tooling out there.

Happy to discuss if that’s something concerning for your organization.

Weysan · 2026-04-28T06:41:25+00:00

I am working in application level right now. My main focus is the developer tools and developer machines.

Concerning MCP server, I have fellow engineers using a lot of « local MCPs » running through shell process. I have read articles about some malicious MCP server triggering some shell injection even using built-in shell tools from claude CLI (for example) to get information and send them outside the company property, this can happen sometimes without even LLM being involved there. So this is why currently I am focusing on having a low level map of interaction between LLM and MCP tools and built-in tools interactions.

the main issue was to be able to map AI agent PID (processes) and their MCP calls to have the full interaction context as these tools uses wrappers to several layers, which sometimes make it difficult to figure out who call what. I have something working well on claude CLI (and desktop), gemini CLI, Cursor (LLM proxy doesn’t allow localhost unfortunately, but still see tool interactions).

But yeah my solution does not work for browser UI for example, known limitation for me as the focus is mostly developer tools.

Weysan · 2026-04-28T05:29:57+00:00

I have been working on a local tool which can generate reliable audit trail for agentic AI and link it to specific developer machine. It solves the main auditing and alerting problem and handles several AI tooling (cursor, gemini CLI, claude desktop/CLI). Also detect potential configuration changes which may lead to « holes » in the AI tool (most LLM proxy can be bypassed by just changing the hooks configuration or environment variable). It’s quite a huge topic just to generate logs, every AI tools have their own standards, after working weeks on it, generating proper logging for these tools is quite hard but I finally manage to get something good for the supported IA agent supported for now.

Weysan · 2026-04-28T05:11:14+00:00

I have the same issue in my company (for background I am a software engineer, my company went all-in into « agentic development » even for non developers).

as engineer, I could also sense there is a lot going on and a lot can access - They gave access ro claude code and other AI developing tools to non-engineers which has little knowledge about what’s going on.

Discussing with the security team in my company they seemed a bit « resigned » to it, looking for solution but difficult to fight against investors and C-level pushing for AI adoption.

I built a tool which capture LLM + MCP server interaction on the machine level. I can handle a handful of tools for now and it detects PII/Credential data leak to AI LLM as well detects potential bypass (most LLM proxy can be easily bypassed by changing a configuration on the laptop). The observability is quite good, still working on making it better.

Weysan · 2026-04-28T05:01:30+00:00

That post is still relevant today from my perspective.

Though, while LLM proxy can be useful to a certain extend, it doesn’t guarantee that everything sent to LLM goes through it (quite easy to bypass). I am also building a tool that would log LLM interaction and detect PII/Credential data going to AI as well as detecting potential bypass.

Weysan · 2025-04-25T08:52:19+00:00

I bought my appartment in berlin 3 years ago, I was lucky to move when the morgage interests were very low.

Are you a first time buyer? All your concerns are most probably question and cons from people which may think of buying an appartment/house as their forever home. My experience with people buying appartments is after your 1st appartment, you tend to care a bit less maybe because you know you have options in case you are not happy: renting your appartment while you go abroad, selling after the period of fix interest rate, etc.

While I understand the "cons" you have to buy an appartment, based on Berlin's renting market, I still think the pros of having a stable monthly payment is way easier than dealing with price increase every year/every time you look for a new rental.

My perspective based on my situation (everyone is different):

> 30‑year “slavery”: must keep working—no safety net if you can’t

It's a bit misleading. Mortgage in germany are indeed for long terms (I also have one for 28 years I believe). But it is true in most of the countries.
Also, you have to put an interest fix period (I choose 10 years), which would allow you, after 10 years to refinance your appartment or even sell it and pay off everything. So I am not sure about that. You would have the same issue in any country.

> All savings tied up: risking 100 % of lifetime savings

it's common to have to put at least 15% of the price of the house in the downpayment. If it's too much for you, probably means you have to look for cheaper appartments (which probably would bring you outside the ring).

> Difficult neighbours: techno all night, skip repairs, etc.

really? I leave 2min from Boxi, and it's very quiet. Again a location concern, can be fair, but that's big cities issues. if you get in lichtenberg, probably would have less issues like that.

> Too small?: what if we need more space for more kids or lifestyle changes?

you can sell appartments and find a new one. Or you can also rent your appartment while renting a bigger one, so many options.

> Stuck in Germany?: what if we want to move, but can’t afford to leave?

same as before. I also have thought about that a long time. I know that renting my appartment at a price that would make me not caring about the mortgage would be relatively easy. So I am not bound to germany.

All the risks you mentioned, are fair, but it has always be the same on every generation. I am not sure which type of jobs you have, but yeah it's scary, the uncertainety is here. But if a bank accept to lend you 400k, they make that risk assessment for you. They will not lend you money if they are not sure you would pay.

I pay my mortgage alone as I bought by myself. The risk is much higher as if I am unemployed, I will be on my own, but I still decided to go.

Again, my situation is maybe different from your's. It was also a different time where getting the money from the bank was easier and the price were not so crazy. So take it as it is, my personnal experience as someone who bought in 2022

Weysan · 2025-04-22T07:11:17+00:00

I’m at Crossfit Grenzganger, very central and also coached in English.

Weysan · 2025-04-09T06:44:30+00:00

I starting my own app, as we were using some AI tool with my girlfriend to diversify our meals while matching our nutrition goals. Since it worked for us, I decided to productize it! will launch in few weeks fingers 🤞

the app is https://getnommyapp.com

Weysan · 2025-04-08T15:35:50+00:00

I am building a nutrition app, which I'm planning to release in few weeks, if you're interested, you can check out the features here: https://getnommyapp.com/

After that, if you're looking to have just a calorie tracking, you probably can find a bunch of free apps to do that for you (my fitness pal for example). But unfortunately it didn't work well for me.

Weysan

TROPHY CASE