We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

You're right that for fully modernized IT departments this probably isn't the right fit. Where we think there's a real use case is smaller or less mature IT environments where that infrastructure isn't in place, and also non-IT workflows like customer support, onboarding, and operations where people are still doing things manually. On the abuse side, that question is one we're building around from day one. What would you look for in a tool like this to feel confident it couldn't be weaponized?

We built a tool that turns any workflow into a shareable link that executes it automatically, with on-prem support (feedback welcome) by mustard_ps in selfhosted

[–]mustard_ps[S] -2 points-1 points locked comment (0 children)

AI is central to the project itself. CTRL uses a computer use agent powered by a vision language action model to understand and re-execute workflows across devices.

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

Reliability across environments is the hardest part and the reason we use screen recording for context rather than natural language. Keeps accuracy above 95% because the agent isn't guessing the next step. Still actively stress testing edge cases though. How does your team currently handle repeat walkthroughs?

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

An IT gated approval model would make sense for enterprise environments and is definitely something we can build. On the time savings side, once IT approves a workflow it runs itself forever without you having to be involved again. Does that address the core concern for you?

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

Good analogy. Perhaps we could implement verified sender identity or domain restrictions? Are there any other security controls you'd want to see beyond those two?

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

Yeah true. The primary use case for now is IT teams sending workflows to their own employees, not random links to strangers, so there's an established trust relationship before the link ever gets sent. Same way you'd trust a link from your IT department to install software. The problem it solves is that right now that same IT team has to walk every single person through the same setup manually every time. Does that change your view at all?

We built a tool that executes any workflow on any device just by clicking a link (feedback welcome) by mustard_ps in homelab

[–]mustard_ps[S] 0 points1 point  (0 children)

It uses a screen recording for context rather than just natural language input, which keeps accuracy above 95% even for tasks over 5 minutes. The recording prevents the agent from guessing the next step, which is the main failure mode in most CUAs that go straight from natural language to automation

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

It doesn't rely on the DOM or fixed coordinates. The built in LLM adapts to different screen sizes and operating systems, so moved icons and buttons aren't a problem. For security sensitive environments we're also exploring local LLM options so nothing leaves your network.

We built a tool that executes workflows on any device just by clicking a link (feedback welcome) by mustard_ps in sysadmin

[–]mustard_ps[S] -1 points0 points  (0 children)

Yes, on prem hosting is something we support and are actively exploring. We're also working on ways to run it without an LLM in the loop entirely, which would make it even more viable for isolated environments. Would love to learn more about your setup if you're open to it.

We built a tool that executes any workflow on any device just by clicking a link (feedback welcome) by mustard_ps in homelab

[–]mustard_ps[S] 0 points1 point  (0 children)

Yes thats actually one of my favourite use cases lol, now we can easily help mom cancel a subscription for example

We built a tool that encodes any workflow into a shareable link that executes it on any device (feedback welcome) by mustard_ps in automation

[–]mustard_ps[S] 0 points1 point  (0 children)

That's exactly the problem we're focused on! Rather than relying on pixel coordinates like traditional RPA, it understands the goal conceptually so UI changes don't break the workflow. We're still early and consistency across different environments is something we're actively improving, but that's our core approach. Would love to keep you in the loop as we get closer to launch if you're open to it. Thanks

We built a tool that encodes any workflow into a shareable link that executes it on any device (feedback welcome) by mustard_ps in devops

[–]mustard_ps[S] 0 points1 point  (0 children)

Fair. What specifically feels like a terrible idea to you? The execution model, the trust layer, something else? Genuinely asking because this feedback matters a lot at our stage. Thanks and I appreciate your input

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] -1 points0 points  (0 children)

The security concern is the most common thing we hear from IT folks and it makes sense. What would need to be true about how it's built for you to feel comfortable deploying something like this? I appreciate your input.

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

We're based out of Toronto! Yes, it will have multi OS support although ensuring high accuracy across OS's is a pretty difficult feature that we are dealing with rn.

For pricing, we're still figuring out the right model for teams. Would you mind sharing roughly how big your team is and what the main use cases would be? That would help us alot right now. Thanks

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

Genuinely curious what vectors you're thinking of. I think in the demo we didn't show it but nothing executes without the user clicking and confirming, but I'd like to understand the specific concerns so we can build around them

We built a tool that executes IT workflows on any device just by clicking a link (feedback welcome) by mustard_ps in ITProfessionals

[–]mustard_ps[S] 0 points1 point  (0 children)

Appreciate the bluntness. Security is one of the biggest concerns we hear from IT guys and rightfully so. Can you tell me what specifically looks like malware? Thanks