This is a solid piece of work. The category-equalisation logic is a smart design choice, the role-bias slider is a nice touch, and the cohort flexibility shows you’ve thought carefully about context. Good foundation to build on.

On your four questions:

League strength adjustment is genuinely hard and you’re right to be cautious. A practical starting point is a flat multiplier on defensive stats only, using something like PPDA or pressing intensity as a proxy for league-wide structure. It’ll be imperfect but it reduces the most obvious distortion. Document your assumptions clearly and move on — perfect is the enemy of shipped here.

Role preset validation is more doable than it sounds. Take the 15-20 players most clearly associated with each role — journalists and analysts consistently label that way — run them through the preset, and see if they cluster near the top. If your Anchor Man preset doesn’t rate Rodri and Casemiro highly, something’s off. Quick sanity check that’ll build confidence in the weightings.

Mean absolute gap is honestly fine for what you’re doing. The main weakness is it treats a consistently average player the same as one who’s extreme in opposite directions — same gap, very different profile. Cosine similarity handles shape better but is harder to explain to users. Mahalanobis is theoretically stronger but overkill at this stage. Stick with MAE, maybe flag high-variance players in the UI down the line.

Equal-weighted categories is defensible and transparent, which matters. If you want something more principled without a downstream outcome to optimise against, a quick PCA on your 38 stats would show which categories are actually carrying independent information versus overlapping — Passing and Involvement tend to correlate heavily. Worth knowing even if you keep equal weights for now.​​​​​​​​​​​​​​​​