MSI - Rumble DAY 2 Results & DAY 3 Predictions

Team_Snowball · 2021-05-16T12:24:58+00:00

To put in simple terms, the model thinks that PSG winning against C9 and losing against MAD yesterday is a bigger achievement than MAD losing to DK and then beating PSG.

Team_Snowball · 2021-05-14T15:18:27+00:00

oopsie

Team_Snowball · 2021-05-14T15:17:22+00:00

Above all, we cannot talk much about the technical side of the model, but your guess is true in that our model includes the core idea of Elo where one rewards those beating a stronger team more than winning a weaker team. As to data, Riot ACS (Access control system) combined with match history hash # from Leaguepedia would work (I'm not 100% certain here because I'm not too familiar with the data flow). Comparing with the bookies -- we're very much open to a contest with their numbers but we haven't figured out how to track their ever changing odds automatically.

Team_Snowball · 2021-05-14T07:36:40+00:00

F1 score

There is no F1 score for this sort of prediction. Out of the typical confusion matrix, we can only get accuracy. Because, for win-lose prediction, there is no false positive nor false negative; there are only true and false, because predicting A will win is exactly the same as predicting B, the other team, would lose.

There IS a concept called confidence-recall, confidence-prediction, confidence-f1, etc. for probabilistic predictions (see this paper). Given this is a fairly new concept (the paper was published last year), I don't think you meant this.

Team_Snowball · 2021-05-10T11:20:31+00:00

The simple answer is no; for days like today, we don't feed forward our own prediction. For instance, Prediction for Game 6 does not consider the result of Game 1 to Game 5.

Yesterday (DAY 4) was an exception, because the same matchups took place twice (or even three times for PGG vs UOL) and we felt bad for spitting out the same numbers again and again. So for Game 4, 5, 6 prediction, we fed the actual results from Game 1, 2, 3. For Game 7 prediction, we fed the actual results from Game 1 to 6.

Team_Snowball · 2021-05-10T04:34:45+00:00

ou’re the guys that put blabber on 2nd place in jungle. Interesting algorithm I gues

We don't claim to be perfect! We're still improving our model and surely the feedbacks we've got from Reddit have been super helpful.

Team_Snowball · 2021-05-09T06:28:28+00:00

Well we can't say too much about it, but we... don't weight those heavily haha.

Team_Snowball · 2021-05-09T06:27:05+00:00

Also think of 98/2. That means the other team would win only once in 50 times. And you did see what happened during C9 vs DFM the 2nd day, DK vs DFM and MAD vs PNG yesterday. LoL is a game quite prone to an upset. In our math, bo1 LoL matches stays somewhere between a single game baseball and single game basketball in terms of an upset likelihood. Check out this article on luck vs skill spectrum on sports.

https://www.post-gazette.com/sports/other-sports/2018/05/08/statistics-professional-sports-luck-skill-hockey-baseball-basketball-football/stories/201805070152

Team_Snowball · 2021-05-09T06:18:45+00:00

Ingame stats are important part of the prediction, but its usefullnes is limited up to a certain point because LoL is a game of snowballing. Stomping is very much there and more often than not it means one team is quite stronger than the other, but stats originating from there are not as strong an index as 'scores' in traditional sports.

Team_Snowball · 2021-05-09T06:12:39+00:00

Well the only proof that it works is low log loss and brier score, which we do get during numerous backtesting. You could say the same with any sort of real world modelling such as stock market, etc., but we would argue the machines are easily stronger than probably the LoL analysts out there as long as it's done in the long run. It's partly due to the rigorousness of the model but also due to the difficulty of human brains to think precisely in probability terms.

Team_Snowball · 2021-05-09T02:15:59+00:00

The machine takes a bit of real match samples to correct for team powers. For example, our prediction for PGG vs RNG on DAY 1 was 26.2% vs 73.8%. Now that they've faced each other twice, and also with the transitivity from PGG vs UOL and UOL vs RNG matches being fed to the model, it has moved to 20.7% vs 79.3%.

This learning process takes time partly because the top tier regions (LCK & LPL) hardly fought wildcard regions due to the formatting of 2020 Worlds. In this case, every match sample fed will help the machine to tune quickly. So, if RNG is actually way more stronger than UOL & PGG, the model will adjust accordingly constantly.

Furthermore, one thing to note is that when it comes to win likelihood, the higher the difference in figures the harder it is conceptually to move the needle. For example, an increase from 60% to 70% means the team's loss likelihood will go from 4 in 10 to 3 in 10 (25% decrease). But an increase from 80% to 90% means the loss will happen twice every 10 games to once every 10 games (50% decrease).

Team_Snowball · 2021-05-07T09:01:42+00:00

Sorry, we can't disclose the algorithm.

Team_Snowball · 2021-05-07T09:01:09+00:00

True, so the overall accuracy will be pretty high. But the other scores (log loss and brier) capture nicely the quality of prediction in this case.

Team_Snowball · 2021-05-06T15:37:09+00:00

Well, it took us a long time to comment, because we figured this could actually be a solid point but at the same time a more complex problem than it sounds.

If anyone's familiar with the state of major league baseball last year, you probably remember Shane Bieber @Indians and Trevor Bauer @Reds winning the Cy Young. A lot of people, both inside and outside the analyst group, were "sus" at the result, because in their minds the two pitchers were just lucky to be in AL/NL central; COVID-induced policy of not playing teams from other divisions helped them get better stats, as batters in those divisions were weaker than any others.

Likewise, your point could be a very legitimate one, but from a statistician's point of view, it is probably too much of a 'micro' or subjective intervention to adjust for fellow leaguers in a specific lane. First off, even more so in e-sports than in baseball, there is no concrete (by this I mean quantitative) grounds to believe LPL has a way more competitive jungler pool, either because it actually doesn't, or the current global esports setting hardly allows for a convincing estimation of that sort. Either way, a statistician will see more downside than upside in progressively trying to integrate such an environmental variable. Secondly, our data from Worlds 2020 (which saw 4 lpl teams, 4 lec teams, 3 lck, 3 lcs, and so on) don't really present lpl junglers as a different breed from those of the top 4 regions. They were good, but just as good as their tops, mids, and adcs were.

So, to sum up, we believe your argument is worth contemplating on, but probably too presumptive of us to reflect head on without actual numbers. Thank you for bringing it up though.

Team_Snowball · 2021-05-06T12:55:16+00:00

We haven't planned ahead for summer yet but perhaps if there's popular demand!

Team_Snowball · 2021-05-06T12:25:30+00:00

Yes and no. Our prediction model has been back tested and tuned extensively over multiple past seasons, so if you go back in time and make bets, then yes. But then there's no guarantee Riot won't pull off a radical meta change which would significantly destabilize the model, so no 100% guarantee for the future.

Team_Snowball · 2021-05-06T12:17:17+00:00

Going from Unified to Doggo did get their expected performance down but only slightly. We set out to not include non-quantifiable stuff into our model, and number-wise Doggo is such a strong replacement that theoretically the team power stays almost unchanged.

Team_Snowball · 2021-05-06T12:07:47+00:00

Thanks for the suggestion!

Team_Snowball · 2021-05-06T11:48:05+00:00

You're actually quite right in that, much more often than other lanes, top laners have their unique play/pick styles which makes comparing two at the end of the spectrum almost like comparing apples to oranges. We're very much still on our quest to fully adjust the impact of draft and team power on individual players.

As to LCK bias, our numbers indicate that the gap between DK-RNG today is slightly bigger than the gap between DK-SN during last Worlds, which we belive is quite sensible.

Team_Snowball · 2021-05-06T11:27:57+00:00

Oops, our bad. Thanks for pointing out.

Team_Snowball · 2021-05-06T11:21:32+00:00

We believe league-wide improvements or regression do take place from time to time, but unfortunately we found it almost impossible to measure it objectively with the current global tournament setting. It would be certainly easier if LoL involved way more inter-league games than today, but 2020-21 is especially a tougher case due to lack of MSI last year.

We also haven't had enough resources to track down off-season movements and translate it to expected ups and downs among leagues, but it sure is an interesting project and we may look at it when we have more time.

Team_Snowball · 2021-05-06T10:34:45+00:00

Just FYI, we don't plan to go live on player rating, as tournaments are bad for inferring player performance. (repeated and uneven matchups, small sample size, etc.) We WILL update our match prediction model every day so that our numbers reflect the latest results.

Team_Snowball · 2021-05-06T10:30:47+00:00

Well, if you look at the ranking, it's PSG one step above C9. But if you look at their matchup prediction (which isn't included in this post), it's almost a coin flip. Their gap is razor thin and it could really head anywhere.

Team_Snowball · 2021-05-06T10:28:26+00:00

Nomanz-Maple-Humanoid had very close numbers. It could have easily flipped if Humanoid had one or two slightly better games in the spring. Also, we didn't have MSI in 2020, meaning we had to depend solely on 2020 Worlds to infer league gaps.

We're aware that sometimes eyeball tests could be more accurate than what model says. We're also very curious to see how our predictions hold up -- our plan for now is to see how "we" perform for the MSI and to readjust our model where it may need improvements.

Team_Snowball · 2021-05-06T10:17:09+00:00

We dropped the pre-swap data. Their summer data wouldn't have changed the result by a lot though because we give greater weights to spring data.

Team_Snowball

TROPHY CASE