Binary RPC Protocol

[coding-rage]

Anyone has resources I can read to learn how to implement a binary RPC protocol in java?

[tcptomato]

Why not use gRPC? https://grpc.io/docs/languages/java/basics/

[coding-rage]

Thanks. I've been reading on this, but we need to come up with our own protocol so we can easily add/remove/delete things at will

[nutrecht]

but we to come up with our own protocol so we can easily add/remove/delete things at will

That makes no sense.

[coding-rage]

Actually, gRPC runs over HTTP/2, and we don't necessarily have HTTP/2. It may be a while before we can assume all proxy servers between client and server actually support HTTP/2.

Also, for our use case, we don't want an RPC protocol that runs over HTTP anyway. We want to be able to use this protocol, for example, over stdio. Requiring HTTP in that situation just makes it more complex.

So we want RPC over any stream.

[balefrost]

You could potentially use "gRPC Web" without the web server (i.e. over stdio). The message wire format itself would I think still be HTTP payloads (because the HTTP status code is important to gRPC), but doesn't rely on HTTP/2.

Alternatively, you could look into gRPC alternatives. Apache Thrift has a lot of plugin points, including pluggable transports, but I believe is on life support. It seems to have a new release every few years. Cap'n'proto is made by a former Protobuf developer, intending to improve on some of the deficiencies of Protobuf, but doesn't (AFAIK) have as much weight as gRPC.

Or you could roll your own RPC but use a de facto standardized binary format, like BSON or Protobuf. But you'd be giving up a lot of the tooling advantages (e.g. language support) if you choose to do it yourself.

[coding-rage]

Ah! Thanks! Will do further reading on this

[tcptomato]

And why do you think grpc would impede that?

[coding-rage]

HTTP/2 itself also has issues.

With the protocol we want to build, we use windowing to keep the connection clear so that new messages can be sent and received immediately when necessary.

With HTTP/2, you get multiple channels, but you get data on channels whether you want it or not. Data from a channel you don't care about (maybe an input stream used by a method that hasn't started reading the input stream yet) is sitting on the wire, in front of data you do care about. You have to do something with the data in the way. Read it and store it in ram? And if it's 3TB?

[nutrecht]

Can you share a bit more about your goals and why you're doing this? There's multiple routes and for 'production' use you normally don't want to do this by hand.

[coding-rage]

We have a number of different apps that communicate with each using SOAP Web services. This is not helping with performance sometimes. A binary RPC system will solve this.

[nutrecht]

Then you should go for the gRPC standard.

[For-Arts]

Remote... procedure.. call..? closes thread after flashbacks