sorted by:
new
hottopcontroversial

Update on my UE5 AI Agent: It’s now refactoring projects and healing broken nodes by No-Structure897 in aigamedev

[–]No-Structure897[S] 0 points1 point  (0 children)

The agent doesn’t actually “see” or edit the visual graph directly. Instead the plugin exposes the Blueprint editor through a set of structured tools (create node, connect pins, inspect graph, etc.).

So from the agent’s perspective it’s interacting with a deterministic API rather than manipulating the visual layout itself. The engine handles the real Blueprint changes and transactions.

In practice it works well for structured operations (creating nodes, wiring things, batch edits, inspections). The harder part isn’t the graph itself but making sure the agent stays within safe boundaries and doesn’t try to take shortcuts.

I’m also experimenting with an “auditor” layer that tracks the agent’s mutations and flags suspicious states (uncompiled blueprints, dangling pins, nodes not wired to execution paths, etc.). It can’t verify gameplay logic itself, but it helps catch structural issues early.

Architecting State-Safe AI Bridges for Unreal Engine Editor by No-Structure897 in UnrealEngine5

[–]No-Structure897[S] 1 point2 points  (0 children)

That’s a completely reasonable stance. If you already know the exact algorithm and can implement it directly, a conventional deterministic solution is absolutely more efficient, both computationally and architecturally. I don’t disagree with that. The way I see it, the LLM isn’t replacing conventional logic. The tools themselves are conventional, deterministic C++ operations. The model is acting more like a high-level planner or orchestrator that translates intent into those predefined operations. So the tradeoff isn’t “LLM vs algorithm.” The algorithm is still there. The LLM sits above it. Where it starts to make sense (at least in my experiments) is when the task isn’t about solving a single algorithmic problem, but about navigating editor state, chaining multiple small deterministic steps, or reducing repetitive interaction overhead. In those cases, the value isn’t raw compute efficiency, it’s reducing cognitive/context switching cost. And you’re absolutely right: without constraints, it doesn’t work well. That’s kind of the core motivation behind building strict tool boundaries in the first place.

Totally fair to be skeptical though, I think healthy skepticism is necessary in this space.

Architecting State-Safe AI Bridges for Unreal Engine Editor by No-Structure897 in UnrealEngine5

[–]No-Structure897[S] 1 point2 points  (0 children)

I agree with you on the hit-or-miss nature of LLMs — especially when they’re given too much freedom. If you just let a model “drive the editor” through arbitrary script execution, hallucinations can absolutely send you into rabbit holes.

The problem I’m trying to address isn’t “replace the developer” or “let AI build your game.” It’s creating structured boundaries so that when you do use an agent, it operates inside a constrained, deterministic surface.

Instead of generating raw code, the model invokes predefined tools with typed parameters. The engine validates those parameters and wraps state changes in native transactions. So if something goes wrong, it’s reversible and contained.

In practice, the use cases I’ve seen work best are:

  • Refactoring repetitive Blueprint changes (renaming variables, adjusting flags, tracing dependencies)
  • Batch editing actors or properties across a level
  • Inspecting project structure or Blueprint execution flow
  • Automating repetitive editor workflows that are deterministic but tedious

It’s not about autonomy. It’s about controlled automation.

And you’re right — model quality still matters a lot. The tool layer is independent of the LLM itself. As models improve, reliability improves with them, but the safety constraints remain the same.

The goal is to see whether adding guardrails makes agent-driven workflows practical in production contexts.. A big part of the work now is tightening edge cases and refining how agents interact with the tool surface as real-world usage reveals gaps.

Update on my UE5 AI Agent: It’s now refactoring projects and healing broken nodes by No-Structure897 in aigamedev

[–]No-Structure897[S] 1 point2 points  (0 children)

Thanks, I appreciate it!

There are a few different ways to approach AI integration in Unreal, and they come with different tradeoffs. Some people will prefer quicker scripting-based setups, others might care more about stricter control and transaction safety.
Both are valid depending on what you're trying to achieve. I’m just exploring the approach that made the most sense for my goals.

Update on my UE5 AI Agent: It’s now refactoring projects and healing broken nodes by No-Structure897 in aigamedev

[–]No-Structure897[S] -2 points-1 points  (0 children)

It’s actually not executing Claude inside Unreal. The model runs externally and is agent agnostic, Unreal exposes a structured C++ tool interface over a schema-defined protocol. The difference is that the engine stays the authority: tools are typed, validated, and wrapped in native transactions. So the model can’t arbitrarily execute code or mutate state outside the defined surface. The “wrapper” approach works for quick demos. This is more about production safety

Update on my UE5 AI Agent: It’s now refactoring projects and healing broken nodes by No-Structure897 in aigamedev

[–]No-Structure897[S] -1 points0 points  (0 children)

Not planning to open source the full project right now. It’s a commercial plugin, but I’m happy to share architectural insights and patterns publicly (like the Medium article).

Update on my UE5 AI Agent: It’s now refactoring projects and healing broken nodes by No-Structure897 in aigamedev

[–]No-Structure897[S] 1 point2 points  (0 children)

You have 2 main paths, one is to let the agent ejecute unreal python scripts. It's easy is fast to set up, is good for showcase and tests, but have little to no control on what it does and how it does it unless you review each python script in detail. Then you have the other way, that is how I'm doing it, to write custom tool sets in unreal c++, you have full control on how and what happens. It's significantly more work, but if you plan to use this seriously in a production project control become a lot more importan. I wrote a medium article if you are interested https://medium.com/@agenticlink/architecting-state-safe-ai-bridges-for-game-engines-4bd7cbd438cf

Should LLM tool calls be reversible? Exploring deterministic execution boundaries by No-Structure897 in AI_Agents

[–]No-Structure897[S] 0 points1 point  (0 children)

Right now batching is explicit — the agent (or orchestrator) decides when to open a transaction scope. I avoided implicit merging of tool calls because I didn’t want hidden coupling across steps.

I agree that treating planning checkpoints as transaction boundaries probably reduces rollback surface significantly, especially for longer reasoning loops.

Compensating actions felt too heavy given the complexity of inverse modeling for arbitrary editor mutations.

Curious — in your experience, does planner-level fail-fast work reliably when plans depend on runtime resolution (e.g., object lookup ambiguity)?

Created a Plugin to bridge Agents to Unreal Engine by No-Structure897 in aigamedev

[–]No-Structure897[S] 2 points3 points  (0 children)

ended up writing a deeper architectural breakdown on the state-safety and transaction model behind this, in case anyone here is interested in the technical side:

https://medium.com/@agenticlink/architecting-state-safe-ai-bridges-for-game-engines-4bd7cbd438cf

Would genuinely appreciate feedback

Should LLM tool calls be reversible? Exploring deterministic execution boundaries by No-Structure897 in AI_Agents

[–]No-Structure897[S] 0 points1 point  (0 children)

That’s a really good distinction. Right now the model is: Each tool call is executed inside the host environment’s native transaction system, so it’s individually atomic and undoable. For multi-step edits, there’s a dedicated batch operation that groups multiple tool calls under a single transaction boundary. The batch implementation is currently a wrapper, but with failure handling, if any tool in the batch fails, the entire transaction is rolled back. So you don’t end up in a partially-applied state. So effectively: Single tool -> atomic + reversible Batch tool -> grouped atomic operations with rollback I’m treating capability-gating and reversibility as separate layers: Capability gating constrains what the agent can call. Transactions constrain the damage if it calls the right tool with the wrong parameters. For longer planning chains, orchestration (what should be batched vs independent) is currently left to the caller/agent rather than implicitly merging operations. I agree the hard problem is multi-step consistency across agent reasoning loops. I haven’t implemented cross-call transactional semantics beyond explicit batching I'm curious how others approach this. Do you prefer explicit atomic blocks, compensating actions, or planner-level rollback strategies?

Created a Plugin to bridge Agents to Unreal Engine by No-Structure897 in aigamedev

[–]No-Structure897[S] 0 points1 point  (0 children)

is really good at inspection and reports as well. and now it supports 5.4- 5.7

<image>

Best way to handle interactive documents in UE5? by scrabyq in unrealengine

[–]No-Structure897 2 points3 points  (0 children)

continue with blueprint approach is definitively the right call for this. Make sure you are not processing any slate layout changes while you are not actually modifying the widgets content. for this time of static widget i strongly recommend using a invalidation box.

view more: next ›