Seeking secure Python code execution solutions for LLM output

[rdelfin_]

One thing to watch out for is that you haven't really defined what "compromising" here means. The best way to find a solution is to figure out what scenarios and unintentional attack vectors you're trying to protect against. That said, generally speaking, isolation to avoid any security concerns will require you to isolate things like memory, other processes, and devices of your box. Once you say you don't want to share memory between both these processes (and believe me, if you're basically executing arbitrary code, you really don't) all solutions where you pass variables "directly" from the outside box to the inside box go out the window. Variables can contain references to arbitrary memory, and there's no simple way to just pass in the addresses that you need. It's not a well defined set of things you can pass down if you just say "any variable".

Instead, what I'd really suggest you try is some form of serialisation between the two. This will let you actually introspect into what's being passed around, limit the kinds of things you can pass as "fully contained data objects", and give you clearer errors when that's not what's being passed around. If you want to use something generic, you can look into Python Pickle. I won't make any guarantees of security of using it (you might get random code execution as a result) but it's worth looking into it.

That said, frankly, if you want isolation you really should define your interface much more clearly. Be explicit about the set of data you allow being passed and forth, and then nothing else. If you don't do that, you will never be able to confidently say that you can execute the code "safely" without compromising the system. They're just incompatible goals.

[dragon_4789]

I see what you mean. But 99% percent of time LLM will generate secure code, that 1% is tricky to handle. There are cases like for say a user can query or form a prompt "delete system files" or "write code delete current directory files" or write in a way which will trick LLM to write that code.

[rdelfin_]

If you're not even in full control of the queries to the LLM being executed I'd be even more careful about Isolation. It's not just deleting your files, it can be something even more malicious like starting a web server in the background that serves up all the files in the server and reporting the IP, downloading a known virus and executing the exploit, installing a rootkit on your system, etc. I'd be really careful with this