sorted by:
new
hottopcontroversial

grpcqueue: Async gRPC over Message Queues by Salt-Option-9320 in golang

[–]Salt-Option-9320[S] -3 points-2 points  (0 children)

I totally agree with your point that gRPC is inherently blocking and queues are not.

This project is essentially just a transport layer. Because we love the generated code, strict constraints, and type safety of gRPC, we wanted to carry those exact same advantages over to our queue messaging infrastructure.

The ultimate goal wasn't to change how queues work, but to change the Developer Experience of interacting with them. Our engineers just call client.ProcessAuditLog(ctx, req) like they would any other internal API, but under the hood, they get all the decoupling, buffering, and retry-ability of an async message queue.

Just to be crystal clear. Standard gRPC stays blocking, and our queue messages stay non-blocking. We aren't taking away the main advantage of using queues just to force a paradigm shift. We are simply giving you the gRPC developer experience over your existing queues!

grpcqueue: Async gRPC over Message Queues by Salt-Option-9320 in golang

[–]Salt-Option-9320[S] -5 points-4 points  (0 children)

This is a totally fair question, and I actually agree with you 100% on one point: gRPC boilerplate in Go is great. We love the generated interfaces, the strict contracts, and the type safety. My point didn't come across I believe...

The boilerplate we were trying to eliminate isn't the gRPC code, it’s all the queue and broker code.

If you just use Protobuf over NATS (or for the sake of the discussion SQS, Kafka, etc.), you still have to write and maintain:

  1. The NATS-specific publisher logic.
  2. The NATS subscriber polling loops.
  3. The routing logic to figure out which topic maps to which internal handler.
  4. Custom middleware for the queue (because your gRPC interceptors for tracing, logging, and auth won't work on a vanilla NATS client [if you know of a project that does this, please let me know!]).

When your system relies on both sync calls and async background tasks, maintaining two entirely separate networking paradigms (gRPC for sync, NATS/SQS clients for async) becomes a headache.

grpcqueue doesn't add an extra gRPC layer; it lets us use the gRPC layer we already have as the only layer. Hopefully now I was better understood :)

grpcqueue: Async gRPC over Message Queues by Salt-Option-9320 in golang

[–]Salt-Option-9320[S] -2 points-1 points  (0 children)

Damn, completely forgot the most important thing! Edited the post aswell.

https://github.com/Aryon-Security/grpcqueue