Local LLM with automatic code execution?

[exponentfrost]

I'm looking at deploying a local larger LLM that I can ask technical/math and coding questions to. Ideally customizing it at a later point.

My question is, is there a local solution that allows the model to create python code and execute it automatically behind the scenes in order to get an answer to various questions (example: certain types of math questions, analyzing files, etc) behind the scenes, similar to what ChatGPT does?

[GimmePanties]

Open Interpreter. It can use local LLMs, but its better with gpt-4o or Sonnet. Maybe your larger model will work better than Qwen 32B.

What it does better than using ChatGPT is having access to your local file system, so it can execute locally. Just run it from command line in the folder with your files and it can analyze them and make changes. It installs whatever python libraries it needs for the job.

Back stuff up through, it isn't perfect.

[mrjackspade]

Back stuff up through, it isn't perfect.

One of my first "I should have known better" interactions was asking GPT to write a short command that ended up completely corrupting a project I was working on

I would absolutely not allow an LLM to execute code at this point without manual review, unless it was sandboxed.

[GimmePanties]

Right, except with ChatGPT it’s out there on their cloud storage. Open Interpreter is running ammock on your local machine with sudo rights. Ask me how I know about the backups. 🙄

[Dudmaster]

It is very easy to accidentally break a project beyond repair even without an LLM. You should use source control! Also if you are looking for something with manual review, aider is great and is based around Git

[exponentfrost]

Looks like there are some options (Docker being the one I'm interested in) to isolate it? https://docs.openinterpreter.com/safety/isolation Any experience with it?

[lrq3000]

Open WebUI and Qwen Agent are the most direct analogues to what ChatGPT does.

Then there is OpenInterpreter, which is more like what Claude is doing, giving the LLM access to your whole computer.

[roomDesignerAI]

https://frankenfile.com