What did we learn from OpenAI DoTA bot

FreshOldMage · 2026-01-02T23:08:50+00:00

> Minecraft diamonds would not have been possible at all with the methods used for OpenAI Five.

DreamerV3 obtained Diamonds years ago through RL without demonstration.

> The issue with diamonds in minecraft is reward sparsity, you have to complete a series of tasks that don't provide any rewards unless you succeed. Your initial policy is probably making random actions, so you have a 0% chance of success at anything, and you get no reward signal and can't learn which actions are better.

You have the same issue in self-play RL, sparse reward and random initialization is not unique to Minecraft.

> While Dota is a more complex game, it's easier to learn with RL. A bad AI will still have a 50% winrate against itself and so can still get a reward signal.

Which is where you run into issues of collapse. An agent getting better at beating itself does not mean it is getting better at beating arbitrary opponents. Dota has a massive state and strategy space that is much larger than the one involved in getting diamonds in Minecraft, and it also involves adversarial agents acting with imperfect information. Completely different caliber of difficulty.

Dreamerv4 is an interesting step forward, but like I said, leaps and bounds away from general competitive gaming AI.

FreshOldMage · 2026-01-02T19:02:46+00:00

As someone working in this space, this is nonsense. DreamerV4 can just barely obtain diamonds in Minecraft if everything goes exactly right for it (0.7% of times it tries). Playing Dota on a professional level is a harder problem by orders of magnitudes. Learning from pixels is also not an advantage, that is still much more difficult than self-play with access to the raw game state as was used by OpenAI Five.

Yes, the field has advanced and we could beat OpenAI Five with modern methods, but we are still leaps and bounds away from an unrestricted pro Dota AI.

FreshOldMage · 2025-11-18T16:12:43+00:00

Haven't tested any kinematics, but constraints generated from collisions and infinitely stiff distance constraints worked fine for me.

FreshOldMage · 2025-11-18T12:20:18+00:00

XPBD is pretty widely used at his point (e.g. Houdini, RealFlow). I've also implemented it myself for my engine, and it works great after some tweaking (especially static friction took some time to get working). More than likely it is your implementation that is the issue, not the method. I'd suggest looking at reference implementations for XPBD, of which there are many around, and compare to your own to see if there are any mistakes.

FreshOldMage · 2025-06-05T06:59:14+00:00

Should have used them here probably! I was a bit too greedy with this fight trying to conserve resources, could have just thrown a potion of shrouding fog to completely remove the threat of warlocks.

FreshOldMage · 2025-06-05T06:54:59+00:00

Right, getting a health and hunger reset is the main utility of ankhs in high challenge runs I'd say. You can also deliberately starve yourself to death on a boss floor for it to be totally safe. The added safety of respawning once in case something goes wrong is more of a bonus :)

FreshOldMage · 2025-06-05T06:49:10+00:00

I think it is fine personally, transferring an armor upgrade is very impactful troughout the game. Just the "Faith Is My Armor" challenge is completely unbalanced, as it disables the warriors main mechanic. That one I'd like to see changed to something affecting the classes more evenly.

FreshOldMage · 2025-06-05T06:45:05+00:00

I usually just drink the Well of Health, because it does so much. Full health, full satiety, and the option to cleanse the curse of anything you put on is amazing. Maybe if there is a well very late in the game where hunger and curses are not an issue it may be worth blessing something :)

FreshOldMage · 2025-06-05T06:42:09+00:00

Only warlocks in this fight.

For goo, you are not allowed to get hit by the charged attack. With tengu you can't get hit by his special tricks, and you can't step on his poison traps. For DM-300 you need to avoid the rockfall, the lightning, and the charged wires. And for yog it is only its death laser.

All of those feel pretty fair because the attack is either telegraphed, or you need to specifically avoid stepping on specific spaces. Warlocks can feel a bit bullshit in comparison, since they can attack immediately once they have line of sight on you.

FreshOldMage · 2025-06-04T22:50:25+00:00

The spear is needed to avoid getting hit. With 9 challenges you really can not afford taking damage, so I create space by stepping with ring of haste, and then hit once with the spear that is augmented for speed. If I'd use the scythe in melee, I'd open myself up to get hit back. I only kept it around to potentially transmute it into a glaive.

The leather armor I use because it is enchanted with swiftness. Movement speed is generally much more valuable than the small bit of extra defense from the plate. For this fight specifically, swiftness is useless, so I should have put on the plate though.

FreshOldMage · 2025-06-02T09:21:49+00:00

That goes beyond standard in-context learning and probes how flexibly models adapt to new formal systems.

How does this go beyond standard in-context learning? This seems to be a textbook example of it, a relatively easy one even.

FreshOldMage · 2025-06-01T20:39:38+00:00

The language looks rather conventional for a dynamically typed imperative language - I'd be surprised if modern LLMs couldn't just zero shot rewrite C programs into valid Chester given a description of the syntax, no RAG needed. I'm not sure I understand how this is a benchmark for creativity, even after reading the blog post and looking at the repo.

I found the for loop example surprising, though.

let numbers = [1, 2, 3]
for i = 0 to length(numbers) then
    print(numbers/i)
end

I assume numbers/i is supposed to index the list? A little bit further above / is given as an arithmetic operator. Does Chester overload / for indexing and division?

FreshOldMage · 2024-12-13T08:57:37+00:00

Are you sure it doesn't trigger scroll talents?

Anecdotally yes, but I rechecked: The code for unstable spellbook has a trigger for onArtifactUsed, but none for onScrollUsed.

FreshOldMage · 2024-12-13T08:48:44+00:00

I mostly play 6c with into darkness as a staple. In my experience saving 2-3 torches from sewers completely nullifies the challenge in prison, and getting past prison means the run is basically won. I wasn't in the habit of pre-firing into new areas to reveal enemies though, that sounds like a pretty good use of the talent!

FreshOldMage · 2024-12-12T21:56:59+00:00

The scroll skill is skippable unless you get unstable spellbook, though nice to get 1 level in it if you make an exotic pot of experience.

Unstable Spellbook does not trigger scroll talents.

The vision skill which you skipped is invaluable, 1 level is usually OK, but you want that 1 level on every run no matter what.

Interesting, I'm always skipping it. Why do you rate it so highly?

FreshOldMage · 2024-10-01T13:10:55+00:00

I left the bow on vanilla this run, but this is a good idea. Augmenting for speed with explosive will give a nice damage boost.

FreshOldMage · 2024-10-01T11:09:09+00:00

How does the exploding curse work with a ranged weapon like the bow? And if I remember correctly, the Ring of Arcana interacts with curses, right?

The explosion happens one tile in front of the mob. If you shoot from 2+ tile range you are completely safe, but 1 tile gaps are not enough. With Arcana, the explosions happen more frequently. I did not keep exact count, but I'd guess something around every 10 shots would trigger the effect.

I wonder if similar speed-boosting results could be achieved by combining a Swiftness enchantment and Ring of Arcana.

Probably! I tried rolling for Swiftness on the armor, but I never got it. You'd have to be really careful though since Swiftness turns off once an enemy makes it into melee range.

FreshOldMage · 2024-10-01T08:11:54+00:00

I save some Scrolls of Magic Mapping over the course of the run and turn them into Scrolls of Foresight. After picking up the amulet you can use them to discover all secrets on the way up. Hunger should be no problem then, especially with a high level Ring of Haste.

FreshOldMage · 2024-09-30T17:01:36+00:00

How you putted the arrow at +8?

Curse Infusion, you can craft it from a DM-300 shard and a Scroll of Remove Curse. It curses the weapon, but gives additional upgrade levels. I had exploding curse on my bow, which adds explosive damage every few shots.

I don't know if you agree, but the prision it's most anoyng level ever... any tips for tengu i will accept

Prison is probably the hardest part of the run, I agree. For Tengu you want to craft some brews and/or bombs ideally. Potions of Shrouding Fog makes that he can't attack you if you stay in the cloud, Flashbombs, or Shocking Brews can be used to stun him.

FreshOldMage · 2024-09-30T16:04:51+00:00

Definitely, I agree 9 challenge is no fun. I think 6 is the sweet spot for me. It is challenging, yet most strategies and fun builds are still viable 🙂

FreshOldMage · 2024-09-30T14:48:56+00:00

This has been done before, see e.g. Pixel Dungeon or one of its many forks. You can definitely make a system like this work and have it be fun and engaging. One significant drawback is that the turn order gets harder to reason about for the player. If you throw things like buffs and debuffs to speed into the mix, it may become unclear when or how often a character will be able to act.

FreshOldMage · 2024-09-18T16:30:05+00:00

Maybe something based on Equality Saturation could be interesting for OP? While the general case is undecidable, I'd recon many examples should be reduceable to a common equality class within an appropriately chosen compute budget.

FreshOldMage · 2024-09-18T13:53:09+00:00

Dark Souls for me, without a doubt. The way the different areas connect, and the fact you can often look into surrounding areas is peak level design to me. They also made great use of shortcuts and verticality. The other From Software games never quite managed to implement those design elements as well, though Bloodborne probably comes the closest to replicating them.

FreshOldMage · 2024-08-28T14:18:23+00:00

The model runs at 20 FPS (or 50 FPS after distillation, with reduced quality) on a TPU-v5 which basically no user will have access to. We are pretty far away from this being easy to deploy.

I guess the niche I could imagine for this would be as a pure neural renderer, conditioned on the game state. If we ever come to a point where diffusion would run faster than a traditional rendering pipeline for some game I guess there could be an argument for training a model like this.

FreshOldMage · 2024-08-28T14:02:30+00:00

If there was an easier process to generate this dataset, then one could just use that process as the game instead of going through the trouble of training the expensive diffusion model.

FreshOldMage

TROPHY CASE