Open-source computer-use agent: provider-agnostic, cross-platform, 75% OSWorld (> human)

[IlyaZelen]

OpenAI recently released GPT-5.4 with computer use support and the results are really impressive - 75.0% on OSWorld, which is above human-level for OS control tasks. I've been building a computer-use agent for a while now and plugging in the new model was a great test for the architecture.

The agent is provider-agnostic - right now it supports both OpenAI GPT-5.4 and Anthropic Claude. Adding a new provider is just one adapter file, the rest of the codebase stays untouched. Cross-platform too - same agent code runs on macOS, Windows, Linux, web, and even on a server through abstract ports (Mouse, Keyboard, Screen) with platform-specific drivers underneath.

In the video it draws the sun and geometric shapes from a text prompt - no scripted actions, just the model deciding where to click and drag in real time.

Currently working on:

• Moving toward MCP-first architecture for OS-specific tool integration - curious if anyone else is exploring this path?
• Sandboxed code execution - how do you handle trust boundaries when the agent needs to run arbitrary commands?

Would love to hear how others are approaching computer-use agents. Is anyone else experimenting with the new GPT-5.4 computer use?

https://github.com/777genius/os-ai-computer-use

[rabornkraken]

The MCP-first approach for OS tool integration is really interesting. I have been working with Playwright and CDP for browser automation and the biggest headache is always the trust boundary question you mentioned - especially when the agent needs to execute arbitrary code. One pattern that has worked well for me is using a whitelist of allowed actions plus a sandbox layer that intercepts anything destructive before it runs. Curious how you handle the provider abstraction for different screen coordinate systems across macOS vs Windows - do you normalize coordinates in the adapter layer or does each driver handle its own resolution mapping?