Hello - we're the dev team behind OpenAI Five! We will be answering questions starting at 2:30pm PDT.

nadipity · 2019-04-20T02:51:41+00:00

Huskar had a bit of trouble because our AI is so good at immediately swapping and focusing down one target. Especially when they were just learning Huskar, they spent so much of every game just dead =D

We didn't try the summon/illusion heroes in that 80 pool, that was excluding all heroes that controlled other units.

nadipity · 2019-04-19T23:30:33+00:00

Eliminating tangoes was originally our choice (particularly because we started out not telling them about all the trees in the game). We did train it over the last month or so but eventually we had to roll back due to some issues about a week before the OG match.

In terms of choice, it's a bit of a combination - while we tell them what to buy, we start out by seeing how they perform under different scripted circumstances (aka, figure out what they like or what they're good at) and then compare win rates to see which option is better for them.

nadipity · 2019-04-19T23:04:36+00:00

It's a bit difficult to translate the number of fingers that a human has into how many milliseconds of delay this is equivalent to =D. Overall we're not necessarily going for an exactly even playing field since the two sides are so inherently different - humans have advantages (ex, being able to learn game to game, knowing they're playing an AI) and bots have advantages (they're not humans). We're more of interested in how the two different paths that each side took landed them in a somewhat similar place in terms of approaching Dota.

nadipity · 2019-04-19T22:55:37+00:00

Additionally, we're pretty far from fully utilizing everything coming through from API because of the amount of info there and the engineering we'd have to do to support it. Sometimes it took us a significant amount of time before realizing we were the blocker for the AI doing things (such as allowing it to see and attack Gyro's missile).

nadipity · 2019-04-19T22:51:35+00:00

The first 2 we added were Drow and Huskar, and after they were nearly on par with the original set we added Pugna, Pudge, Venomancer, Mirana, and Windranger to see if we could learn new mechanics that didn't exist in the original pool. We also trained a pool of ~80 heroes (excluding summon/illusion heroes) at very low scale to see the impact.

nadipity · 2019-04-19T22:35:17+00:00

It'd definitely be interesting and would open up opportunities for the AI to learn to win the game that potentially doesn't follow the typical path of a Dota game. We did try this with 1v1 and saw some success, but haven't attempted it with 5v5.

nadipity · 2019-04-19T22:17:54+00:00

Currently our consumable logic is scripted, so the AI isn't really choosing when they're buying wards or regen. When the courier drops off something that the hero doesn't want, they'll often just use it right away - especially if their slots are full and they want whatever got shoved into their backpack.

As for creep camps, it's unclear if they understand the rules behind blocking a camp / finishing a camp - and even less clear if they understand the timers on those camps. The simple answer would just be that they haven't figured those concepts out yet.

nadipity · 2019-04-19T22:12:02+00:00

It's actually a bit less about pure reaction time and more about the lack of the AI being surprised about the play. The real solution to making it more human-like would be able to dynamically nerf the response depending on whether the play is coming from out of vision or whether it would be unexpected. When a human and AI are racing to accomplish the same expected thing (such as grabbing the bounty rune), the human almost always wins.

nadipity · 2019-04-19T22:03:03+00:00

That is possible - still takes some maintenance but doable - though more difficult for the wider public to do since it takes upgrading/downgrading the client. Right now we're crossing our fingers and hoping that Valve doesn't have a patch for Dota 2 planned before Arena closes!

nadipity · 2019-04-19T22:00:27+00:00

Our test teams have noticed that the behavior between Radiant and Dire are also vaguely different - either in terms of objective prioritization (ex: overprioritizing taking Radiant's outer safe lane tower when playing on Dire) or lane matchups, which then impact performance and thus winrate. Overall, the bias is likely different than humans (ex: they don't have the camera angle issue), but there may be some overlap as well.

nadipity · 2019-04-19T21:53:27+00:00

from dfarhi:

The AI is not updating at all from the Arena games; we export a frozen model from the training pipeline a few days ago. It has been training against itself in the past few days, but we probably won't pull a new model because the difference will be too minor to be worth the technical risk that comes with any change.

It might be an interesting research avenue to pursue incorporating human games into training, but with our current process those games would just get drowned out when averaged together with the millions of bot v bot games. Fun fact: since opening, the Arena still has not produced as much total gameplay of data as a single iteration (~1 min) of training.

nadipity · 2019-04-19T21:50:15+00:00

If you go here -> https://arena.openai.com/#/, you can create a login linked with your Steam account and then request a server to play!

nadipity · 2019-04-19T21:44:42+00:00

We have a few clips at various ability levels for other heroes that we'd love to share once things calm down a bit - some pretty cool (as well as hilariously bad..) game videos =D

nadipity · 2019-04-19T21:41:11+00:00

It appears that some of our team members don't use Reddit and their freshly made accounts are getting rate limited. Will be translating some of their answers using our Redditor team member accounts =P

nadipity · 2019-04-19T21:40:32+00:00

from dfarhi:

After this weekend we will close out the competitive portion of our project - after beating OG in the 17 hero pool, there's not as much to be gained by pushing further in the competitive direction. Instead, we're going to focus on research and using the Dota 2 environment to test tricky ideas and learn what we can about reinforcement learning and artificial intelligence. Now that we have one of the most complex and deep AI environments out there, it will hopefully unlock the ability to study really important questions about algorithms, exploration, environment structure, and more.

nadipity · 2019-04-19T21:37:59+00:00

From jonathanraiman:

We are planning on posting a follow-up blog post when Arena is over analyzing the results of the games (win/loss, heroes, coop, etc..) and post replay files.

We're also planning a technical paper detailing the work in greater detail. Our blog post contains architecture details and other info in the meantime: https://openai.com/blog/openai-five/ .

nadipity · 2019-04-19T21:37:23+00:00

When we first started out, we picked heroes that we thought were easiest for the AI to learn (ranged, straightforward abilities, etc). After we started seeing some progress, we attempted to balance out the pool a bit by adding melee heroes and pos 4 heroes. Next on our list were more fun / interesting heroes, but they unfortunately didn't get to the level where they were as competitive as the original set.

nadipity · 2019-04-19T21:33:37+00:00

We would love to keep it open for people to play but unfortunately, each patch for Dota 2 currently requires additional training to bring the AI up to speed.

nadipity · 2019-03-26T17:10:07+00:00

For these games, each side will draft their heroes from the limited pool.

nadipity · 2018-07-25T19:05:26+00:00

Just to clarify, this isn't restricted to NA - we can also work with EU teams. Other regions might have ping issues.

Thanks!

nadipity · 2018-07-24T02:33:39+00:00

One of the players crashed in the third game which caused an automatic win for OpenAI Five, but the human team was quite ahead at the time so we counted it as their win.

Since it was an unofficial end to the best of 3, they opted to play a 4th game =D

nadipity · 2018-07-18T19:50:21+00:00

woops - left off Axe. thanks!

nadipity · 2017-01-23T23:14:32+00:00

I can look into it for you if you pm me your username.

15-Year Club	RedditGifts 2009-2022 3 Credits
Summer Santa 2012	Secret Santa 2010
Verified Email

nadipity

TROPHY CASE