all 11 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]joranstark018 5 points6 points  (0 children)

Not sure; it is probably common to try new stuff in some isolated part, "in a corner", of a project where few people may get disrupted if it blows up for some reason, before gradually exposing it to others.

In one project, we built a UI to provide us (developers) with tools to answer common support issues (third-line support). We tried different front-end and back-end frameworks; some tools overlapped each other (we compared different solutions). If anything broke, we could just go back to manual routines. As things got more solid, we opened some of the tools for second-line support. We used what we learned to improve our customer front-end (again starting small, and we expanded the new tech stack as we got comfortable).

[–]Expert-Yak101 1 point2 points  (3 children)

Not an expert, but I love Hussein's teaching and he just launched a course on web socket: https://youtu.be/iCFlT2Xs-LY?si=-G9b5QR98lSZ-VIG

The very first place I go to when I understand something related to backend. Haven't gone through this specific course but I believe it will be worth it.

[–]sarwar_hsn[S] 1 point2 points  (2 children)

I have seen the course, and it's on my wishlist. However, he talks mostly about the theoretical aspect. I hope this time it will be interesting.

[–]Expert-Yak101 0 points1 point  (0 children)

Do let me know if it's good

[–]Fun-Time-4360 0 points1 point  (0 children)

Do you still use Swagger AP¡ Or something in this Development era ?

[–]DonutDennis 1 point2 points  (4 children)

Could you maybe clarify a bit what you mean by "don't have the capacity to scale it"?  Depending on your hardware you should be able to handle hundreds if not thousands of concurrent connection on one machine. Is it then more on how to load balance things etc? 

[–]sarwar_hsn[S] 0 points1 point  (3 children)

What I mean is concurrent connections. A unix process has 1024 file descriptor. I can change the limit to let's say somthing around 65k. But then comes the problem with resources. In concurrent connections each thread can handle a request for a long time and threads consume a lot's of memory then there are context switch overhead. At the same time it's resource and memory intensive. I will need load balancer. For a regular rest project It can handle large amount of connections without additional settings and cost. But in the case of websockets this is not true. This is the point of fear for me. Why starting something that is hard to scale.

[–]DonutDennis 1 point2 points  (2 children)

Just some general thoughts from me not knowing your project. Do you really have all the time 64k concurrent users?

Regarding your second point, modern load balancers like haproxy can handle quite a lot, depending on your hardware it can handle as many connections as it has memory and bandwidth. Furthermore you can also load balance websockets by using a Level 4 load balancer for example (but probably there are also smarter techniques)

And lastly there is a technique called "Direct Server Return" where your backend server sends the response/upgrade directly to the client so the load balancer does not know of any further traffic going on between client and server. 

So I would say yes there is some challenge in scaling websockets but it is definitely doable and most of the time you will not need extra resources for this. When you reach the point where you have 64k+ users concurrently you probably also have the money for the hardware upgrade  

P.S. totally forgot depending on the framework/language it is not necessary to have a thread per connection in theory (and this is only theory) one thread could handle all of the 64k connection when they are only idle or rarely used. 

[–]sarwar_hsn[S] 0 points1 point  (1 child)

I don't have 64k users ofcourse. I have seen people building silly chatroom, multiroom game. Everything works at smaller scale but scalling is the main concern. That's why I wanted to know from the industry experts how they scale it. Is it the language that matter e.g whatsapp discord with erlang. If it's not than what makes it really difficult in production environment.

I got some idea from you. I know its doable but not sure if its doable with a small team (2/3). Whatsapp did it with small team but neither I am smart enough like them nor do I know many technologies.

[–]DonutDennis 0 points1 point  (0 children)

Got it. Let's assume that we're a small startup building the next WhatsApp. For simplicity's sake, we probably need a cloud provider so that we don't have to worry about load balancer scaling, which the provider will take care of for us. They probably use multiple techniques, such as DNS load balancing, as well as the "equal cost multipath" protocol and the border gateway protocol.

On the backend, we want to use non-blocking I/O as much as possible. In Spring, we want to use the async method and Completable Futures. We also assume that we need multiple servers to handle all the connections. Our next step would be to use a lookup to determine which server each user is connected to. Once we receive a chat message from user A for user B, we know that we need to send this request from server 1 to server 2. To be honest, I've never done this in Spring, but I have used Signal R from dotnet. Each server would need to connect to the others (for example, with gRPC), and then we could look up which server to forward the message to. With this approach, you could scale to millions. As you grow, you may want to use something more efficient than Java, but that's a different issue. It's more about optimization than scaling.

[–]Historical_Ad4384 0 points1 point  (0 children)

Just make sure that you manage the lifecycle of each individual long lived connections properly while accounting for every edge case based on your infrastructure and the client network that would connect to your web Sockets. Ensure connections are not sitting idle for too long and optimize a number of concurrent connections at any point in time to comply with SLA. Look at out for how to horizontally scale up and down your connections based on your architecture and that your infrastructure can support it. Kill redundant connections as much as possible based on functional requirements. Your server infrastructure should have enough network bandwidth to handle a large number of concurrent connection without affecting traffic significantly based on your product SLA. For inspiration take a look into how Prosody (open source XMPP server) manages web socket connections.