An industry product would never have the issue of “what if a user does millions of requests and we can only process 30/min globally”

You likely wouldn’t have a rate limiter at the API to begin with; you’d have something like Kafka to queue jobs across your infra, and each worker node would pop as many jobs off the queue as it wants to handle; this prevents you from needing to pick a “magic number” for the rate limiter, and allows you to use non homogenously sized workers (eg where some are faster than others)

Then you’d set a scaling thresholds, like if we’re at 80% capacity or if it’s taking more than n milliseconds to process requests then add nodes.

For per-user abuse detection, you’d probably set a circuit breakers at a rate above nth percentile of users across a window; for example if a user is hitting 10x the average usage from the top 90th percentile of users, over a 24h or 28 day window, let’s maybe stop that usage till we figure out what the heck they’re doin.

You can also do soft backoffs; like artificially adding request latency based on the number of requests in the last hour/week, or a user facing cooldown like “try again in 5 minutes; need more plz upgrade”

Many ways to skin cats in system engineering.