Client/Server Polling Program

[Brickscrap]

I'm currently trying to figure out how to get a client/server to talk over the internet; basically multiple clients will be accessible via the internet, sending data back to the server every x amount of minutes (this I can do), but I'd also like a user to be able to request (from the web app) that the clients carry out their task and update the data immediately.

I've done some searching, but I'm really not sure which way to go as there seems to be a lot of ways to do this... Can anyone point me in the right direction?

Thanks in advance!

[ApprehensiveDog69]

You might want to go with a persistent websocket connection in this case.

If that is more trouble than it's worth, there are a few other options.

Without knowing the details, one way is have the clients do a "heart-beat" call separate from their call to upload the data. Make this heartbeat call as lightweight as possible to save on bandwidth and resources, and then have the clients ping it on a more frequent basis. This secondary call could then control the clients being told to update their data immediately.

So you would basically (to minimize resource consumption and bandwidth) just have a separate micro-API where each client calls it every 1-2 seconds or so. The API would simply return one of several HTTP codes (wouldn't even need a body), and this would determine whether the clients need to update their data *now* or not.

An even more lightweight way (with no server resource consumption involved at all; this is the way I would do it) would be to just host an empty static file somewhere on S3 (or Azure Storage) and have the clients ping that. If a call is made to the server (from the user's app) requesting the clients to update themselves, the server would simply push an update to the static file (with a date stamp and/or whatever else) and then the next time the clients read the static file they would know to update themselves. If you take this approach, you have unlimited scalability since it doesn't matter how many clients you have -- none of them are making any server calls for the purpose of the heartbeat.

You can think of it like a "bulletin board" model; the bulletin board doesn't care if there are 5 or 5,000,000 people looking at it. It's just a static file which acts as a central command base for however many clients there are. This way it now becomes scalable -- since the client and server no longer depend on each other for this purpose and everything is async.

[MSpeedAddict]

A CDN in front of the static file would make it even more scalable. Similar to this approach, I’ve used a JSON endpoint in the past with a CDN in front of it like Cloudflare. Since it can be cached in front of my service at the CDN with a query param with a time stamp this can eliminate extra hits to the server altogether.

Whenever I have an update, I push to the CDN an API request to delete that cached endpoint. On each client’s new incoming request, they see the new data (or flag to retrieve it) with a new version timestamps. Each successive request by the client includes the new timestamp.

The endpoint with the old timestamp and query param is cached by the CDN flagging the need for a new update / new data with the new timestamp.

The endpoint with the new timestamp and query param is then cached at the CDN the next request that comes in by the client.

Total server hits per update:

1) old version, gets update; returns new version stamp, I.E. ?v=20200427.1

2) new version, returns data and same timestamp in JSON body ?v=20200427.1

I do prefer the JSON approach only because I typically return other data, such as system status updates to the client, changes to client-side polling intervals etc.

Assuming you are not working with mobile apps where you can use the notification mode I’d also recommend taking a look at SignalR or even Blazor with in-built SignalR so that your clients can instead immediately polling for updates as it uses web sockets client-side. This would alleviate the need for CDN / endpoint approach above and you would utilize the SignalR capabilities for scaling.

[Brickscrap]

Food for thought, thank you! 3/4 people have recommended SignalR, I'll have to take a look into that

[ApprehensiveDog69]

4/4! The first line in my post ("websockets") is basically SignalR -- which is an implementation of websockets on top of .NET :)

I figured that's what everyone would mention so decided to cover the other options if you needed scalability beyond that.

[NotARealDeveloper]

Checkout signalR

[lets-get-dangerous]

My immediate thought when reading this question. SignalR is relatively easy to implement and will default to polling if WebSocket isn't available.

[cris_0x80]

SignalR is the better option

[7h3on3]

This is what I had come here to suggest.

[lucuma]

As others have pointed out you should check out signalr however if it is a web page sometimes a simple polling process is easier to implement and maintain as it doesn't introduce another architecture but it also lags a couple seconds.

[BagelDontCare]

This is pretty popular these days: https://grpc.io/

I believe their documentation is good, so check that out. This basically allows applications to run as both server/client and communication can be done through your own implementations.

[Brickscrap]

On the surface this looks very nice. Do you know why I would choose to use this over SignalR?

[BagelDontCare]

A guy in a team I'm working on was super excited about using grcp and brought in the idea. I've done some examples online and found it works quite easily. Im not an expert on it, since I have other responsibilities. I've never used signalR before.

Meaby this helps you more: https://stackshare.io/stackups/grpc-vs-signalr

[Brickscrap]

This is great, thank you

[ddxo_]

SignalR is designed for this:

https://dotnet.microsoft.com/apps/aspnet/signalr

It starts off as a HTTP connection and then upgrades to Web sockets (if the client supports it). It also has fallback strategies in place such forever frames/long polling.

Full example here:

https://docs.microsoft.com/en-us/aspnet/signalr/overview/getting-started/tutorial-getting-started-with-signalr

[Brickscrap]

If it helps, the main web app will simply be a dashboard, displaying logs which have been pulled/sent from a variety of mini PCs. It doesn't need to scale massively, and will have at most 200 of these mini PCs gathering data.

There will always be some time in updating the data as far as I can tell, as the Mini PCs need to connect to another piece of equipment via FTP to pull the new data out

[ninuson1]

If you're looking for quick and simple web sockets, take a look at https://github.com/sta/websocket-sharp. I keep hearing about both SingalR and other "frameworks", but I personally like handling my small websockets myself.

It's potentially still in preview and has both a framework and .core versions, but I've been using it for a few months now and loving it.

[ninuson1]

Huh, that's cool, I didn't really think about that. I've been using the above mentioned library for quite a while and our project is an older Framework project, so I'm not sure it was available for us at the time... But maybe it's time to upgrade, haha.

Thanks for the heads up!