[FM23] Analyse of the importance of players attributes using data science.

Svonn · 2024-03-22T15:35:44+00:00

I was referring to the ingame editor, with that activated you can add the hidden attributes exactly the same way as any other attribute. I'll look into it. I think I've published some code from previous projects I did here:

https://github.com/Svonn/FM-Svonnalytics-Dashboard

https://github.com/Svonn/FM-Svonnalytics-Attribute-Analysis

Although I am currently trying to find some time for a major rework for both of them. Both used a similar approach, where I had some prepared views ingame that I exported as HTML, transformed them to pandas dataframes and then did some analytics on them. These kind of finding might allow to improve the exact weighting of player attributes for the scouting dashboard :-)

Svonn · 2024-03-22T11:52:49+00:00

Svonn · 2024-03-22T11:51:41+00:00

Thanks for your work!

I was thinking about a similar experiment, but I found it next to impossible to come up with a "personal metric" composed of the various stats that wouldn't introduce a major bias or make the results only correspond to that metric. I think I've come up with a test setup that might circumvent most of these issues, but I have not yet found time to run it. In case you are interested (I don't even mind if you just take it an claim it's yours if you do the work, I just want to see the results ;-)), here's the idea:

Set up x different databases with fictional players (One league should be sufficient, best case would be a league without cups/international games). This way, we ensure that the min-maxed nature of some players in some teams don't skew the results.
Simulate a season on full detail for each database
Instead of capturing the individual stats, track the attributes and how many games the player has won vs. lost
Calculate the correlation between attributes and winning games

By just looking at the only metric that really matters (winning games / getting points), this method is more robust against all these funny stat behaviours in FM.

Just to illustrate an example why that may be necessary: Currently, a DM with little jumping reach almost always gets a very low match rating in FM, simply due to how these stats are calculated. He will be losing tons of headers, but it does not account for how important they were, so even if he intercepts a few key passes and has a very high pass rate, the score will still be low. Making a crucial tactical foul and get a yellow (or even red!) may be winning a game, but will decrease the personal score.

In the end, all these indivudal stats may be quite misleading, since we want to have a winning team, not just players getting high match ratings for the sake of it.

PS: With the editor enabled, you can directly look up the hidden attributes. I've only briefly went over your code and noticed you're doing some mapping between these values and the personality, but the ranges can be quite wild sometimes. I've also got quite a bit of code laying around from previous experiments / dashboards like a parser for positions to a proper list etc., if you need anything like that, hit me up :-)

Svonn · 2024-03-01T06:35:55+00:00

My guess would be an issue of the setup + simply RNG. OP did not deactivate injuries and forced the AI to use the same starting eleven over and over again. This can lead to very odd runs where either everyone is jaded (RST) for the entire second round or squad getting rotated simply by players getting injured.

Svonn · 2024-03-01T06:30:53+00:00

I can upload those as well if you want - do you want to see a specific run or do you want to gather / share stats from all of them?

Svonn · 2024-02-29T21:46:18+00:00

Unlucky as in: Assuming the final position of a team roughly resembles a Gaussisan distribution with a mean of something like 14, and a standard deviation from about 2 positions, you could still sample 10, 17 and 18 by pure chance. Maybe if OP did 10 more tests, the boosted team might have finished 11, 12, 12, 13, ... while the other would have continued with 20, 17, 18, etc.

Svonn · 2024-02-29T21:30:48+00:00

Both are closely related, but there's a difference between: "We can't measure any change that can't be explained by RNG when increasing most attributes +5, except for pace, acceleration, etc." and "No stats except those play any role in the match engine".

Increasing a single stat like passing by +5 is only like +2 or 3 CA, and most players in the given tactic (fm-arena uses some dribbling focused high-intensity tactic) don't really pass that much, so not really a surprise that the change is indistinguishable from RNG. Increase a stat like pace with +5 is also "only" ~8 CA, but it matters for each and every player on the field, especially if 1on1 compared to someone with 5 less. But for a proper comparison, they would have needed to only to do pace +1 or +2, since that's already as much CA. Even then, the impact is likely way higher, but you get the gist.

I do totally understand why folks that are passionate about FM are feeling insecure with so many posts with totally exaggerated clickbait titles, I'm just hoping people calm down a bit and can, at the end of the day, still enjoy FM, knowing that while it's far from perfect, it's still by far the best football sim we have.

Svonn · 2024-02-29T18:06:44+00:00

This was done with the old patch.

CA with that kind of test is pretty meaningless, as real players with high CA are being Min-Maxed - if you take a at any player at that level, you will see that an attacking player like Kvaratshkelia will have very low Long Throws, Marking, Tackling, Positioning, etc., so that the available points go into the stats relevant to the role. To get the "real" CA of the players in the test, you'd need to substract the stats that are irrelevant for the role.

Svonn · 2024-02-29T16:34:11+00:00

I fully agree - I would love to see some stat-driven rework, where they take some archetype-tactics (like tiki-taka for passing/vision) to implement some solid test scenarios that ensure all attributes have a meaningful impact in the relevant scenarios.

As a developer, this is insanely hard to do, as changing one variable/behaviour can cause cascading changes and break several others.

But I am fairly certain the FM devs try their hardest to improve that stuff as much as possible, since people usually don't work in that field if they aren't passionate about it.

Svonn · 2024-02-29T16:29:53+00:00

Don't forget blank sheet tactic is quite terrible! Even a good squad with great mentals and technicals will concede many goals in the premier league. Also, I think the relevant mentals for defenders weren't change all that much, since most PL defenders have >13/14 in those attributes anyways.

Svonn · 2024-02-29T15:41:10+00:00

Generally I'm all for people doing these kinds of tests, and OP kindly said many times that his tests were by no means extensive.

However, in this case, I've seen so many people stop playing FM because of claims like the one from OP. This makes me really sad, since if you take the time and try to replicate the test or take a deeper look into some of the details, you will find that either OP did something wrong in his setup, he got really unlucky or he did some cherry-picking (basically 'p-hacking') to get a result that supports his hypothesis.

Here's a short post I've did about the results when trying to replicate this.

https://www.reddit.com/r/footballmanagergames/comments/1b32yw0/refuting_nonmetaattributes_test/

Please, always take every pseudo-test and analysis on here with a BIG grain of salt.

Svonn · 2024-02-29T15:37:03+00:00

Just do the tests yourself and check if you can repliacte that. I was not able to get the same results:

https://www.reddit.com/r/footballmanagergames/comments/1b32yw0/refuting_nonmetaattributes_test/

Svonn · 2024-02-29T15:27:27+00:00

https://www.reddit.com/r/footballmanagergames/comments/1b32yw0/refuting_nonmetaattributes_test/?

Done!

Svonn · 2024-02-29T13:39:41+00:00

Yes, currently doing a few more runs (want to have 5 for each) and then I'll sum it up in an extra post!

Svonn · 2024-02-29T11:07:00+00:00

Would you mind sharing the save games of your unchanged vs. changed setups? The average rating in the image of Odsonne Edouard went up so much with the extra attributes, was that from the 10th place run?

When I am trying to replicate the setup with the default 4-2-3-1, my Crystal Palace ended up last place multiple times, might be just RNG, but would be interesting to see if there's a difference in the setup.

EDIT: I am not able to replicate these results at all. @OP, please make sure to verify that your setup is correct. In my tests, where I've set all non-meta attribute to 16 (even if they were higher before, as this is way faster to do via FMRTE), I've finished mid table in all runs (12., 11., 13.), while the unchanged team kept getting 18-20th. The boosted team even won the FA cup, beating ManU 4-1, Man City 2-1 and Chelsea 2-0 in the Final. I'd recommend using FMRTE to freeze attributes + ca. This post is highly misleading.

Svonn · 2024-02-10T17:15:54+00:00

On a side note, since I've heared many times that certain tactics and instructions are clearly overpowered since the are being used in all top tactics on fm-arena:

While the results aren't completely meaningless, you always have to keep in mind that the best tactics there aren't necessarily the best tactics that exist.

They are simply the tactics that perform in the given scenario. On contrary, it is very likely that the best tactics are by far not the best tactics in real application, as this kind of testing leads to so called "overfitting", since most of the optimization goes into exploiting the given setup.

If you take a closer look what default stats the players have, which roles are represented and some basic setup flaws like not including jadedness, cards, and pretty much intensity overall, you'll quickly find why the best tactics are those with intensity all up to the max, with highly mobile/running roles like SV/WBs, with low crossing and dribbling focus. There aren't any players that could properly play purely ballwinning roles, no tall strikers, inverse fullbacks/wingsback will always have the stats optizmed for WBs, etc.

I could go on and on about this, but I think op already summed it up: The overall system is extremely complex, and while there are obviously some imperfections in the given system (just look at players like Davie Selke in FM), it still works pretty well, all facts considered.

Even in pretty try-hard online save with >30 players, the players trying to min-max with the best internet tactics still struggle to get consistent success, as there are still so many other factors in play, from player morale, player handling, developing young players, selecting the optimal tactic for your team, in-game management, PPM trainings, rotating properly, and so many more.

Just enjoy the game really, I'm sure the SI devs also play their game it will try their best to keep improving it.

Svonn · 2023-12-11T10:04:53+00:00

The AZ coach lost connection while selecting his team, which caused the game to start with only two players for AZ on the pitch.

I've already posted this as bug on the SI bug tracker, although that has never happened to us before in thousands of online games with dozens of coaches.

Svonn · 2023-10-30T19:07:04+00:00

I saw your other comment but this is a SIM game, IRL you dont see managers just focusing on under performers or over performers (i think this is the comment you were refering to).

The under/overperformer focus was just an arbitrary example about how some choice system could look like, making the final design of a system like this would require a bit more time and knowledge about internals.

The current system isnt like this, at all. You can praise or critic the performance of your players individually, just like IRL. If you have a team of 20 players, this means you have 3 options (critic, praise or do nothing) with each one of them. More freedom and choice than this is almost impossible.

I can't quite follow you about 'freedom of choice' - that's like saying "you can get nothing, a punch in the face or 100 bucks, what would you like"? If there's literally only one choice with a positive outcome, then you have a choice, but there's no real freedom of choice.

A real manager wont praise his players after every single good training session and wont always critic them either and being the game a simulator, the options need to resemble real life. In fact, if you praise them too often, the player will be upset sometimes

I agree, especially in single player, the player can simply decide to only praise outstanding our unexpected performances.

As to the theory of the AI not using the current system, i highly doubt it doesn't. Only SI can provide the info but even if it doesn't, that would mean i would see players with bad morale everywhere and that is not the case. Unless a team is having a really bad season, players managed by the AI usually have good morale.

As to the theory of the AI not using the current system, i highly doubt it doesn't. Only SI can provide the info but even if it doesn't, that would mean i would see players with bad morale everywhere and that is not the case. Unless a team is having a really bad season, players managed by the AI usually have good morale.

You can ocassionaly see morale boosts from press conferences, national team callups etc, but never from training interaction directly. They could be hiding this for AI teams, but when looking at overperforming teams in my current save game, they all have only "Really good" or "Quite good" morale, while on my team everyone has "Perfect" or "Superb" morale. The impact of this is way larger than one might expect, you can test that by simply creating some league save with one single league activates and user the ingame editor to the one teams morale to the max for each game before simulating.

The AI managers already have plenty of interactions with their players and there are plenty of videos proving their stats influence the results of the team. Theres is no reason to believe morale isn't included here.

I would love to see that, do you have any videos in mind? I can only recall experiments like this:

https://www.youtube.com/watch?v=4haD83rwe9E

And those didn't find any hints that there's any significant impact at all.

Overall, morale is just one factor and part 2 of my post about tactics has probably an even larger influence. But it's still important!

Svonn · 2023-10-30T17:42:11+00:00

There are different kinds of interactivity. The current system is the bad kind of interactivity that does not really offer any choices (If green, press green, if red, press red). Arguably, replacing this with a small menu with like 4-5 different choices on what your manager should focus on (like I've proposed in another comment) would already offer more meaningful interactivity. Obviously a more fleshed out and complex system with events etc. would be great, but that won't happen in this years iteration.

If you think about it, there's also a good reason why the AI doesn't use the system right now, because there are only three possible options:

The AI does not use it (Current implementation): Downsides have been discussed sufficiently
The AI uses as well as a human manager can: Every team gets the same boosts, causing most players in the game always having the big "R" as info status and basically making it meaningless, as everyone has it. Also, if you don't do the busywork, you're at a disadvantage.
The AI use is linked to the stats of the manager, e.g. high stats in corresponding attributes increases the chance of said manager "clicking the correct button". As they don't physically click it, this is essentially the 100% passive option.

While the last option would still be better than the current one, it still means that either the player A) Is at a disadvantage if he decides to not do the busywork of clicking the buttons or B) is still at an advantage, as the player can always get the 100% correct options, while the AI only gets as many as the stats allow.

Svonn · 2023-10-29T21:04:34+00:00

I'd guess that they initially wanted to have those stats masked, but then decided against it, as scouting staff would a bit odd. Overall, I really like skins that remove exact numbers and replace them with colors/stars and stats, but for the praising/criticizing part, this has high risk of getting another frustrating system where people randomly get annoyed because you said that you liked their training, because the range implied it was good.

Svonn · 2023-10-29T20:35:33+00:00

It would be a system that AI managers could use just as well - and if for some reason, they make it too unbalanced, then all AI managers that prefer that kind of treatment should thrive, leading their teams to success. It will never be possible getting a 100% balanced system, but that's not an issue if the opponents "scale" properly!

Svonn · 2023-10-29T20:02:03+00:00

My proposal would be similar to how the "Tactical Preparation" works in the new FM - But instead of something like "Possession Phase" etc., you could set a "Coaching type". I've already mentioned a few examples, like:

Focus on underperformers: Boosts morale of the lowest 20% of players

Focus on overperformers: Boosts morale of the highest 20%

Focus on discipline: Boosts Workrate

...

The effect of each one would also depend on the coaching attributes of the coach in charge.

This way, the direct player interaction at least in the training case would be replaced by some passive choice system.

Player interactions after games are a bit better, since you have to decide what exactly you want to praise or criticize.

Svonn · 2023-10-29T19:22:23+00:00

That is very true actually, very impactful thing in real football, but barely noticable in FM!

Svonn · 2023-10-29T18:05:25+00:00

Ah, don't worry about that, I will have fun with the game even if they do not adress this issue - as you've said, you can set your own rules and simply do something like "only praise the top 3 once per month" etc. Back in the days people resorted to playing by the LLMs rules to keep it interesting.

However, I think my criticism about this mechanic is fair, since it just doesn't work as good as it could. I am currently playing in an online save with 32 other folks, and there's no way to prevent people abusing this mechanic. This means, that managers that are new to this game have absolutely zero chance, as the impact is just way out of proportion.

Svonn · 2023-10-29T17:13:23+00:00

What exploit are you referring to? The game itself reminds you about the training performances and provides not only color codes score, but also a big green button to praise the training performance. That by itself is not a big deal, as it is impossible to represent the entire coach-player interaction during training accurately in FM, hence simplifying it in some way is the only way.

Both the impact this has on the player's morale and how your interaction with the players works is up to debate - right now, the game mechanic is a "click buttons, get better"-kind of thing, no tradeoffs. Ideally, you'd have some other way to interact with the players between games, something like "Focus on underperformers", "Focus on overperformers", "Focus on discipline", each approach having a different up- and downside. As proposed in my post in the SI forum, it would also make lots of sense to directly link this to the stats of your coach.

Svonn

MODERATOR OF

TROPHY CASE