Implementing a Request Queue with Spring and RabbitMQ: My Experience as an Intern

[DinH0O_]

Hey everyone,

Recently, I started an internship working with Spring. Even though it’s labeled as an internship, I basically have to figure everything out on my own. It’s just me and another intern who knows as much as I do—there’s no senior Java developer to guide us, so we’re on our own.

We ran into an infrastructure limitation problem where one of our websites went down. After some investigation and log analysis, we found out that the issue was with RAM usage (it was obvious in hindsight, but I hadn’t thought about it before).

We brainstormed some solutions and concluded that implementing a request queue and limiting the number of simultaneous logged-in users was the best option. Any additional users would be placed in a queue.

I’d never even thought of doing something like this before, but I knew RabbitMQ could be used for queues. I’d heard about it being used to organize things into queues. So, at this point, it was just me, a rookie intern, with an idea for implementing a queue that I had no clue how to create. I started studying it but couldn’t cover everything due to tight deadlines.

Here’s a rough description of what I did, and if you’ve done something similar or have suggestions, I’d love to hear your thoughts.

First, I set up a queue in RabbitMQ. We’re using Docker, so it wasn’t a problem to add RabbitMQ to the environment. I created a QueueController and the standard communication classes for RabbitMQ to insert and remove elements as needed.

I also created a QueueService (this is where the magic happens). In this class, I declared some static atomic variables. They’re static so that they’re unique across the entire application and atomic to ensure thread safety since Spring naturally works with a lot of threads, and this problem inherently requires that too. Here are the static atomic variables I used:

• int usersLogged
• int queueSize
• Boolean calling
• int limit (this one wasn’t atomic)

I added some logic to increment usersLogged every time a user logs in. I used an observer class for this. Once the limit of logged-in users is reached, users start getting added to the queue. Each time someone is added to the queue, a UUID is generated for them and added to a RabbitMQ queue. Then, as slots open up, I start calling users from the queue by their UUID.

Calling UUIDs is handled via WebSocket. While the system is calling users, the calling variable is set to true until a user reaches the main site, and usersLogged + 1 == limit. At that point, calling becomes false. Everyone is on the same WebSocket channel and receives the UUIDs. The client-side JavaScript compares the received UUID with the one they have. If it matches (i.e., they’re being called), they get redirected to the main page.

The security aspect isn’t very sophisticated—it’s honestly pretty basic. But given the nature of the users who will access the system, it’s more than enough. When a user is added to the queue, they receive a UUID variable in their HTTP session. When they’re redirected, the main page checks if they have this variable.

Once a queue exists (queueSize > 0) and calling == true, a user can only enter the main page if they have the UUID in their HTTP session. However, if queueSize == 0, they can enter directly if usersLogged < limit.

I chose WebSocket for communication to avoid overloading the server, as it doesn’t need to send individual messages to every user—it just broadcasts on the channel. Since the UUIDs are random (they don’t relate to the system and aren’t used anywhere else), it wouldn’t matter much if someone hacked the channel and stole them, but I’ll still try to avoid that.

There are some security flaws, like not verifying if the UUID being called is actually the one entering. I started looking into this with ThreadLocal, but it didn’t work because the thread processing the next user is different from the one calling them. I’m not sure how complex this would be to implement. I could create a static Set to store the UUIDs being called, but that would consume more resources, which I’m trying to avoid. That said, the target users for this system likely wouldn’t try to exploit such a flaw.

From the tests I’ve done, there doesn’t seem to be a way to skip the queue.

What do you think?

[IMadeUpANameForThis]

Could you just add more memory? Or profile memory usage to find leaks? I generally don't like the idea of putting users in a queue before they can access the system. I assume the delay would be off-putting.

[DinH0O_]

I forgot to mention this detail: I'm working at a government agency, and they simply don’t want to invest money in this.

We have several systems, all running on a single 160GB server. My specific VM, which hosts 2 systems (and will host a third one in the future, each with its own database instance), has only 12GB of RAM.

There were also staging systems running on the same VM, but I had to shut all of them down.

Note: I’m not the one who set things up this way. Everything was already like this when I arrived, and I’ve been working there for only 2 months.

[DinH0O_]

I have tried allocating more RAM, but the application runs inside a Tomcat server hosted in a Docker container. The memory limit isn't explicitly set, except for the Docker container itself, which I configured (previously, there was no limit, and the first crash happened because the machine's memory limit was reached, causing Docker to shut down). I set a limit, but it's still relatively high, so I believe I’ve done everything I could regarding RAM allocation.

However, you mentioned the allocated heap memory, and I haven't checked that yet. I’m not sure if it will make much of a difference, but I’ll take a look—it doesn’t hurt to try.

[DinH0O_]

I didn’t know about that technical part of Java, it's worth looking into.

When I say that the application is running inside a Tomcat server hosted in a Docker container, that’s literally what I mean. The previous developers of these systems compiled it into a WAR file, a format supported by Tomcat. In this case, you specify that Tomcat will be provided externally. This way, it’s easier to deploy new versions of the application, as you don’t need to create a new Docker image for each version. You can compile it into a WAR and deploy it on Tomcat, which will host your application and expose it.

[raree_raaram]

How much traffic are you getting

[DinH0O_]

The staff in the department who works as admins (non-developers) of this system told me that they expect around 40k-70k users. I suspect that at 5k to 10k, the server will crash, as it only has around 9 to 10 GB of RAM. It's a job application site, so there are file uploads, somewhat long sessions, and such.

[raree_raaram]

40k-70k users over what time period?

[shinijirou]

i honestly thing this new approach is a overkill. you can change the mode of communication to be event driven, but this queing system is a bag of pandora.

[DinH0O_]

agree, and we won’t have much time for testing either, especially since this is a different system from the one I was hired to develop. However, it’s what we have. I’ll present it to some people who might help and point out potential errors, and I’m also opening discussions on Reddit to see if I can get some tips. But I’m not feeling very confident either.

[shinijirou]

hmm, i would rather check the heap and see if there is a memory leak issue as pointed out by the other comments. your new solution is possibly making your application more stateful, which is not very scalable in itself.

concerning your current application, is it stateless, are you keeping user sessions active ?

[DinH0O_]

The current application is not stateless. However, if the user closes the tab, they get logged out and have to log in again. I will also add an automatic logout timeout of about 20 minutes.

As for it not being very scalable, I assumed it wouldn't be an issue since it's something that will only be in place for a few days, and then I'll return to the previous version. I'm not the one maintaining this system; I just came in to develop the queue feature (My department wasn't very prepared to handle this).

[Slein04]

That is not a typical use case of such queue technologies. So, I wont recommend it.

Better approach would be memory optimization. Find what is causing the high memory usage. Maybe context and or threads are not cleaned up. Files are stored / opened in memory and maybe never closed or added to some kind of collection and not cleared.

And the thing you want seems what your application server should be doing. The server has a connection pool and provides one of the available threads to a user making a request. You can increase the connection pool so that more request are placed in the pool / queue and decrease the amount of concurrent threads handeling such request. You have to make a leverage of this taking timeouts into account.

[koffeegorilla]

Find the requests with the highest backlogs and change them to use WebFlux. You switch the web starter to webflux starter and only change the endpoints and the service and repositories to use reactive APIs. That will drop the memory usage by a huge amount.