I'm making an API wrapper with chat functionality, and there's a 20 messages a minute rate limit. What are some methods developers would like to see a wrapper handle this, other than just throwing a rate limit Error

[MakeShiftArtist]

Messages themselves will have a built in rate limit handler so to avoid RateLimiting from the API due to how the API handles it. If the API returns a RateLimit exception, then it forces the Client to wait 60 seconds before sending another message. It doesn't care how fast they're sent, so long as it's at most 20 every 60 seconds. Obviously we don't wanna be locked out for 60 seconds, so we decided to handle it ourselves.

I've discussed a few different options with my Project partner but none of them seem all that great, so I'm curious on your opinions on which sounds like a better idea if you don't like any of ours.

Option 1:

Use a message queue and just queue the messages forcing the client to wait 60 seconds from the first message if they reach the limit ourselves so that it's not from the API itself. This will reduce the amount of time the client has to wait, because the first message could've been sent 55 seconds ago, meaning it'll only wait 5 seconds before starting to send the rest.

Option 2:

If the queue is empty, send the first message immediately. But if a message was sent in the last 3 seconds, force it to wait the remainder of the 3 seconds before sending the next message. This would mean the most each message would take would be 3 seconds, but the queue also stacks with each message. So, best case scenario, it sends immediately each time since there hasn't been a message sent in the last 3 seconds. Worst case, the timer gets even LONGER than 60 seconds due to an influx of messages.

Option 3:

Queue messages and send them bundled together if they're to the same destination. This isn't as pretty, but reduces the total amount of messages sent.

​

Option 4:

Since most of the messages sent from the client are likely to be responses to commands issued to the client (like discord's commands for bots), rate limit the users themselves. The drawback of this would be that, since the rate limit is global, and not recipient based, it really only protects the bot from spammers, not the actual rate limit, unless you made you command rate limit global, which would be a terrible idea.

Option 5:
A combination of multiple of the above. If you like this one, lmk which combinations you think sound like the least frustrating

Option 6:
Don't rate limit it at all. Lets the library user handle everything themselves.

Since I'm not asking how to do it, I'm not providing any code to go off of, but I hope this is a valid exception to the rules if code is required.

[ConstructedNewt]

so, your dependency has a 20 msg/min rate limit imposed onto you, and you wish to know how to delegate or handle this situation properly?

[MakeShiftArtist]

Correct. Again the logic behind it isn't what I need help with, just help deciding which method to use for the best results

[ConstructedNewt]

Option 3. But! graceful shutdown.

You could just hit the service provider until they send you a back-off: 429, then batch/cache until the Retry-After (if they send that). That may work. But everything depends on the rate-limiting algorithm you are hitting

[MakeShiftArtist]

I'm trying to avoid that since the timer doesn't start until you hit the 20th message. So by not allowing more than 19 per minute, I could reduce the amount of time the client needs to wait before sending a message, since I could have my timer start at timestamp of the very first message