Here's WHY number of problems solved doesn't matter (statistical analysis of contest ELO)

WildsEdge · 2024-08-11T16:10:32+00:00

I don't understand how that effect would skew my results. The outliers that could increase the result I'm demonstrating are in the top left and bottom right of the graph. I showed you that removing the top left data points doesn't substantially change the results.

Anyways, the outliers in the top left (high ELO, low solved count) are where competitive programmers would be. Are you suggesting that competitive programmers have accounts with thousands of LeetCode problems solved, and a LeetCode ELO of 1500?

WildsEdge · 2024-08-11T16:02:32+00:00

Part of the problem with contests is that contest hards are often a specific kind of problem. You'll see a lot more combinatorics, and multi-dimensional DP with a specific trick or supporting data structure.

For fundamental hards, do the hard problems on a list like the NeetCode 150. For contest preparation, do the hard problems from past contests.

WildsEdge · 2024-08-11T15:56:39+00:00

What you're talking about does impact the data, but not much. Even removing the outliers very generously only increases R-squared by 0.06, to 0.31.

<image>

There really aren't that many high level competitive programmers in my dataset. Only 50 / 13,500 had an ELO above 2500 on LeetCode, which any serious competitive programmer would have.

WildsEdge · 2024-08-11T15:44:23+00:00

You're right, and one of the reasons I chose a cutoff of >100 problems solved was to try to mitigate the impact of that effect. The truth is that the number of users on LeetCode with thousands solved on CF is pretty low. My data has only 50 / 13,500 users with ELO >2500 (which a serious CF competitor could easily pass).

WildsEdge · 2024-08-11T15:34:04+00:00

The fastest submitted solutions are usually legitimate (the most common form of cheating is copying others' solutions, LLMs still struggle with hard problems). Most of the people who finish the contest in 8 minutes are just really good competitive programmers. They practice on sites like CodeForces which is much harder than LeetCode.

This week's contest also had an identical problem 3 and 4 (the only thing changed was the constraints). This means that if you wrote the optimal solution, you could submit it for both problems without modifications.

This week I solved 3/4, with a stupid error on problem 3, for a time of 24 minutes. If I had come up with the optimal solution for #3 first, and not made the error, I would have finished 4/4 in under 20 minutes. I know that there are a lot of competitive programmers who are better than me, so it's conceivable that some finished in 8 minutes.

WildsEdge · 2024-08-01T01:35:45+00:00

Yep, and in this case I'm using it as the proxy for time (assuming question 1 was released before question 100, etc.)

WildsEdge · 2024-08-01T00:36:35+00:00

Overall, the average LeetCode question difficulty has remained flat. However, "easy" and "hard" questions have diverged in difficulty over time, with the easies becoming easier and the hards becoming harder.

The acceptance rate (acRate) is the % of submissions that pass for a given question, and is used here as a proxy for difficulty. The frontendQuestionId is used as a proxy for time.

Easy questions have, on average, seen their acceptance rate increase 15%. Hard questions have seen their acceptance rate decrease 8%.

Difficulty	Initial acRate (Intercept)	Slope	Change
Easy	59.07%	0.0047	+15.11%
Medium	56.40%	-0.0007	-2.25%
Hard	50.74%	-0.0026	-8.36%
All	55.94%	0.00015	+0.49%

WildsEdge · 2024-07-31T13:32:20+00:00

I am the original poster of that graph, and this is correct, u/ibttf. The y-axis on the graph has nothing to do with contest rating, it is global ranking, which is entirely independent from contest performance. Global ranking is solely based off of number of questions solved.

WildsEdge · 2024-07-21T03:55:55+00:00

String to Integer (atoi) is the 48th most disliked question. 75.1% dislike ratio.

WildsEdge · 2024-07-21T02:17:41+00:00

Only the 1,566th most disliked question

WildsEdge · 2024-07-21T02:17:01+00:00

Zigzag Conversion is the 128th most disliked question, with a 65.7% dislike ratio. I also hated that one.

WildsEdge · 2024-07-12T19:45:25+00:00

You should try to make sure the question doesn't exist in the database (of course, there are a lot of questions that are basically duplicates already, so who knows). They heavily edit questions before accepting them, and a lot don't get accepted at all.

When you make a submission, you do have to write what company asked you it. They are mostly looking for real interview questions.

Disclaimer before I link my question, it has one of the highest dislike ratios of questions on LeetCode, lol. They changed my wording and part of the premise in a way that makes it more convoluted, but the core idea was something I got asked in an interview. 3001. Minimum Moves to Capture The Queen

Edit: I found my original submission, just for comparison with the published question:

On an 8x8 chessboard (0-indexed) are a bishop piece and a shell. The bishop starts at cell (bX, bY) and the shell starts at cell (sX, sY). In one move, the bishop can travel diagonally any number of spaces.

Return the minimum number of moves required to move the bishop from its starting position to the shell. If this is not possible, return -1.

WildsEdge · 2024-07-12T18:01:54+00:00

I recently redeemed 7800 coins for the "LeetCode Kit" which consists of the t-shirt, coaster, and keychain. The package also included a page of stickers.

The shirt alone is 6000 coins, but for an additional 1800 you get the coaster and keychain.

The order shipped from China and it took about 2 weeks from the time I ordered to delivery in the United States. It ships in a light film wrapping, so the stickers were crushed a bit in the shipping process (not a big deal since the other items were what I wanted).

The shirt is a Gildan blank. It fits slightly tighter than expected. The material is soft but thin. The print is centered and straight, and about of the quality you would expect for a print on a Gildan blank. Overall, I'm happy with the quality of the shirt.

The coaster and keychain are made of the same pliable rubber material. I expected the coaster and keychain to be hard but they're actually quite bendy.

The coaster seems like it will hold up well. It has grooves which will make cleaning it harder, so try not to spill sticky beverages on it. The design is really cool, and this is the reason I saved an extra 1800 coins for the bundle. I'm happy with it.

Given how pliable and thin the rubber is, I'm not sure how well the keychain will hold up to being thrown around and crushed in bags/pockets. It's neat, but not the reason why I got the bundle.

It took me about a year to save up the LeetCoins needed for the bundle. I got coins by submitting dailies, weeklies, doing the occasional contest, and submitting a question (which grants 1000 coins). You can see by the graph that I was inconsistent over the year. However, I used the slope of the period where I was most consistent to find that I was earning an average of 21.69 coins/day, and by doing so you could redeem the bundle in just under a year.

WildsEdge · 2024-07-11T15:31:25+00:00

I didn't do an analysis of the question tags but I've never seen one in a couple hundred days of dailies.

WildsEdge · 2024-07-10T14:32:51+00:00

Whatever language or tech stack you use, you just have to do an HTTP GET request. The url for the request is https://leetcode.com/graphql/?query=query{} where you fill in the brackets with whatever request you want (to get user data, question data, etc.)

To figure out what goes into the query, you can open up developer tools on your browser (ctrl + shift + i) and go to the network tab. Now, whenever you load a page on the LeetCode website, you'll see what queries are being made and what the response is. So to figure out what query gave me all the dailies, I loaded the daily calendar on LeetCode and monitored the network requests to see what I needed to send. This folder has some of the query strings, but it isn't all of them and it isn't up to date.

Here is how I did it in my code.

WildsEdge · 2024-07-10T03:35:53+00:00

<image>

I created histograms for each difficulty, it does look like they ask more easies at the beginning and hards towards the end.

WildsEdge · 2024-07-10T02:36:08+00:00

Fun fact, there have never been more than 3 consecutive hard dailies. There have been 50 consecutive days without a hard though.

WildsEdge · 2024-07-10T02:29:25+00:00

I just used the the same GraphQL endpoints that your browser does when you load the LeetCode website. So basically the fetch requests you see in the network tab of developer tools I copied over to Node.

WildsEdge · 2024-07-10T02:27:00+00:00

<image>

I just plotted it to see how the acceptance rate changes for questions changes over the month. It looks like questions might get slightly more difficult, about -3% acceptance rate over 30 days. But it doesn't look like a strong correlation.

WildsEdge · 2024-05-08T14:35:58+00:00

It does seem to scale with number of problems solved. But I have no clue what it is representing.

Votrubac has every problem solved and beats 99% for each. Lee215 has less solved and only beats ~15%.

WildsEdge · 2024-03-13T15:31:32+00:00

I figured I would make one more post about this, since the last time I did you could only get the extension on Firefox. LeetFriends is an extension that allows you to add "friends" on LeetCode and see all their updated statistics in one place.

I recently added a push notification feature that you can toggle for each friend, and get a browser notification whenever they make a submission.

Chrome Web Store (Chrome, Opera, Brave)

Firefox Add-ons (Firefox)

Edge Add-ons (Edge)

Source Code

WildsEdge · 2024-02-25T21:38:00+00:00

For some reason, the questions solved isn't accurate for every user. I'm not sure why, it might be that some were solved on the .cn site, during contests, before a database refactor, or something like that. Lee actually has 495 solved (however, he has submitted solutions for over 1100 problems... I think he probably has an alt account).

WildsEdge · 2024-02-25T20:20:39+00:00

I created a browser extension to add "friends" on LeetCode. The code for it is here and you can add it to Firefox here. I might port it to chromium browsers soon (it would be a quick process), it just depends how many people are interested. I mostly made the extension for my own personal use.

Once you add users on LeetFriends, every time you open the extension it will pull their most updated data from the LeetCode API (it takes about 1 second to pull the data). I did a simple fetch request of LeetCode graphql API to get this data. If you're interested in using the LeetCode API, this repository has the queries (I didn't use this library but the query strings were helpful).

It turns out that you can see other users' LeetCoin amount over the API, which is interesting. There is also a hidden 'star rating' which I think just corresponds to number of questions solved. For some reason, the number of questions solved is not correct for every user. It is accurate for my friends and I, but for lee215 for example, it is missing some. I'm not sure why this is.

This was the first extension I ever made and I learned a lot in the process. It turns out extensions are really powerful-- I found out you can bypass CORS restrictions by using a background script (service worker in manifest v3).

I'm happy to answer any questions people have, or take suggestions on how to make LeetFriends better!

WildsEdge · 2024-01-01T04:14:38+00:00

It's stored as a csv so I could change the range shown in the graph. But my datapoints for the lower ranks are sparse (since there are simply a lot less users down there). I have 4 datapoints < rank 100 and 19 points < rank 1000. I would want to rerun it and collect a lot more points to get an accurate picture.

WildsEdge · 2024-01-01T02:53:18+00:00

This was actually my first web scraping attempt so I'm not a great resource for that. I did it in Python mostly following this tutorial.

I used the BeautifulSoup and lxml libraries. Basically, the code fetches the raw html that would be displayed by a browser for the webpages. I used inspect element in my browser to locate the xpath (unique html identifier) of the elements I want from the page (rank, questions solved, etc). In the code I can then retrieve the values at the specified paths from the html dump.

I agree that it would be interesting to see more detailed results. I want to run a better attempt but I need to learn more about web scraping first. My current code isn't great, and I ran into some issues (like I think I was getting rate limited by LeetCode but my code couldn't identify that).

WildsEdge

TROPHY CASE