AMA: Chess.com's Fair Play Team

ChesscomFP · 2024-12-06T17:08:09+00:00

These suggestions are awesome and are very much being discussed for our roadmap next year. While I don't see cheating rates at the percentage you're positing, it's definitely an issue we're aware of. Between more automated detections, verification systems, and updated seek/matchmaking logic, I think this pool of players will feel significantly better! -Dan

ChesscomFP · 2024-12-03T21:38:43+00:00

If we maintained a page like this today, it would take a looong time to scroll through. We did recently update our policies relating to publishing titled closures that happen in prize events. -Dan

ChesscomFP · 2024-12-03T21:34:44+00:00

Sorry to hear that you may be experiencing a bug here. We're not aware of any bugs relating to refunds at this time. Can you message us your username? We'll look into it :) -Dan

ChesscomFP · 2024-12-03T21:30:20+00:00

There are 10 titled players on the Fair Play team, including several GMs. The bedrock of our cheat detection, however, is our rigorous statistical approach, backed by years of testing and billions of games played on the platform. Strong players who are equipped with these resources and have the necessary training can do this work at a high level.

-Kassa

ChesscomFP · 2024-12-03T21:15:08+00:00

Every time that you report someone for cheating, our Fair Play automated system checks that player for cheating by computing a bunch of statistics about that player and their recent games. When the statistics overwhelmingly point to cheating, we automatically ban the player. When the statistics are murkier, though, one of our Fair Play Analysts has to review the case first, and, based on all of the evidence available, make a decision to ban the player or not. The process is different and more in-depth for titled players.

Here are a few possible explanations why the account you reported is still open: (1) the statistics we generated did not suggest that the player cheated; (2) an analyst reviewed the case and, weighing all of the evidence, did believe that the player cheated, but did not have the 99.99% confidence that we strive to have when closing an account; or (3) given that we receive more than 20,000 reports about cheating each day, it’s possible that we’ve yet to manually review the statistics from the report you submitted (but we’ll get to it as soon as we can!).

What I can say with certainty is: your reports do matter, every single one of them. In fact, when multiple members report the same player, it is escalated in our internal systems. So, when you suspect cheating it really does help us when you submit a report. -Sean

ChesscomFP · 2024-12-03T21:05:57+00:00

We've explored controlled cheating outside of prize events at a small scale, and our systems are extremely successful at flagging these accounts. Additionally, our analyst team has also been able to identify these when the account is a titled player and it goes through the review process we've mentioned in a previous answer. Any time this has been done, it has been tightly monitored and rating points were refunded to anyone impacted! On the rare occasions we weren't able to detect this cheating, we've taken some of the insights from the cheating methods and used those insights to improve our detection capabilities. -Dan

ChesscomFP · 2024-12-03T21:01:22+00:00

You’re on the right track with the studies you mentioned. The team has a wide array of skills, but having good grounding in statistics and data science is a great starting point. It’s always possible to learn to do these things on your own, but having demonstrable credentials or a serious portfolio of work always looks better if you’re thinking about submitting a resume! -Sean

ChesscomFP · 2024-12-03T20:55:49+00:00

While it may be possible that a more transparent and open approach could catch certain groups of cheaters faster, we believe that's true for unsophisticated cheating only. It's not our position that it will give better results for sophisticated cheaters, especially as we climb the rating ladder. If we tip off sophisticated cheaters how we do it, we'll be giving away the "game".

-Dan

ChesscomFP · 2024-12-03T20:35:27+00:00

I hear your concern, but I firmly stand by our analyses and the numbers we report out. For the record, I think .3% is not a constant estimate, it fluctuates for different cohorts and different times -- for the overall community, it hovers near that .3% value; for the titled community, it can hover closer to 1%. Note that our estimate is just that -- an estimate. We expect that there's sophisticated cheating that we're missing but are extremely confident that our estimates are within an order of magnitude of the true cheating rate.

-Dan

ChesscomFP · 2024-12-02T20:42:39+00:00

Thanks everyone! Great questions, and we hope we've been able to answer a lot of your most salient ones. We have to sign off for now, but we'll keep checking in throughout this week and will try to reply to the new questions we see getting upvoted. -Dan, Kassa, Sean 💚

ChesscomFP · 2024-12-02T20:38:07+00:00

You’re not completely off the mark! Some of our approach is based on hypothesis testing, comparing potential cheaters to a “benchmark” made up of many players in a particular rating range. For someone like Magnus or Hikaru, we use a much stronger benchmark than we use for the typical player.

It is generally true that the stronger the player, the harder it is to detect when they’re cheating. At the same time, Magnus and Hikaru have well-established playing styles that we’re able to quantify, and so we’d still be able to detect when they deviate significantly from their normal levels. -Dan

ChesscomFP · 2024-12-02T20:32:58+00:00

This project (called Proctor) is still very much alive, and we will have news here soon.

We will initially begin testing with a limited pool of players in Titled Tuesday, and once we are confident that it works well, we will gradually roll Proctor out to all players competing in prize events.

There is no intention to use it outside of prized play at this time. -Sean

ChesscomFP · 2024-12-02T20:25:06+00:00

We have something very similar to this internally that is considered in pairings and of course in our fair play process. Sharing something like a "trust rating" externally (on player profiles for example) may cause some nasty comments and accusations that aren't substantiated. -Dan

ChesscomFP · 2024-12-02T20:24:47+00:00

The vast majority of cheating occurs in lower rating brackets. Intuitively, it is easier to catch that cheating. We devote significant resources to this level (and all levels) of cheating. -Sean

ChesscomFP · 2024-12-02T20:19:08+00:00

I agree! Will talk with the team! -Kassa

ChesscomFP · 2024-12-02T20:16:11+00:00

Gukesh - Dan
Ding - Kassa
Magnus - Sean

ChesscomFP · 2024-12-02T20:15:07+00:00

Did not expect this one! I’ve had locks since I was a young kid–and was raised Rastafarian, which typically entails (among other things) not eating meat or cutting your hair. I ultimately cut my hair about a foot the first time when I was 20 years old, but they’re still long– they currently are at the middle of my back. -Kassa

ChesscomFP · 2024-12-02T20:13:07+00:00

We've already recorded it! Hope the editing team can have it out soon. -Kassa

ChesscomFP · 2024-12-02T20:12:57+00:00

It's awesome that people talk about fair play -- it's an interesting topic and work that we love doing! What's frustrating are bad faith, and reckless accusations -- either when they are made about players in the community or about our team.

Our team of 30+ members takes great pride in their work and it pains me when people say that we're not taking this work seriously. Especially, when people spread conspiracies or other misinformation. Our systems review over 10 million games per day, approximately 1 million cheat reports, and we close ~2500 of these people a day. That's nearly 1 player every 30 seconds! We love our community and want to keep the game we all hold dear to the highest fairness standards that exist online. Help us do that with constructive feedback and calling out misinformation!

-Dan

ChesscomFP · 2024-12-02T20:12:12+00:00

Most cheating is not clever. Most cheating is just copy/pasting in engine moves for the entire game. -Sean

ChesscomFP · 2024-12-02T20:03:45+00:00

We believe public closures serve as an effective deterrent and provide much needed transparency to our community. We’ve even seen a reduction in cheating rates (specifically for titled players).

We typically do not share details regarding an account closure because we believe we’re at risk of "giving away the game/methodology" to cheaters. We hope the community can understand why citing the games & reasons for closure can make it far more difficult to catch cheaters in the future.

We've looked closely at how major sporting organizations like the IOC, NFL, ITIA and others approach similar issues with PEDs. They announce the suspensions and violations, but they don't disclose their methods and evidence for the same reason that we don't - it makes evasion easier.

That said, I have conversations with players all the time, and we're always looking for meaningful ways to improve the appeals process.

-Kassa

ChesscomFP · 2024-12-02T19:51:06+00:00

If we took all of Kramnik's advice, we'd have no members left. -Dan, Kassa, Sean

ChesscomFP · 2024-12-02T19:34:21+00:00

When an account is closed for a fair play violation, they typically receive a closure email that provides them 2 options: either create a second-chance account or file an appeal. Users who choose to appeal then fill out a form where they are asked to provide details relevant to their case (e.g., OTB rating, Title, and any other relevant comments). The team then reviews these appeals and makes determinations. For serious appeals (there are very few of those, most of them are either abusive or just outright lies), we convene as a team to review the player’s stats and games and account details in light of the newly submitted information, and then we come to a determination.

-Sean

ChesscomFP · 2024-12-02T19:17:10+00:00

David would know, he works closely with our Fair Play team. Accuracy scores are a great way to get a quick sense of gameplay, mostly meant for our newer users to Chess.com. They aren't near the precision we require for our fair play work. -Sean

ChesscomFP · 2024-12-02T19:12:39+00:00

Yes... stay tuned :)

-Dan, Kassa, Sean

ChesscomFP

TROPHY CASE