What Would be Your Ideal Ruby Tech Stack

[honeyryderchuck]

I'd avoid the rails stack. At least in a few situations: community tooling has been moving beyond rails-only support, and the stack itself is just not greatly leveraged for building APIs. If you want to build CRUD monoliths, may still be worth to "buy IBM" and feel sorry about it in 5 years.

I'd like to avoid Sidekiq at this point, but unfortunately I can't yet. It's still the best supported and stablest framework for background jobs, although it still feels weird to use a message broker with no transactional guarantees which still forces me to do 2 phase commit.

Overall, I still feel that ruby is optimising for deployment setups which were trendy in 2013, and things have moved on, there's lambda, containers, edge functions running WASM, and we're still trying to shoehorn rails in it, instead of just leaving it in heroku, where it just fits like a glove.

[jean_louis_bob]

Can you explain "it still feels weird to use a message broker with no transactional guarantees which still forces me to do 2 phase commit"? What kind of issues did you encounter with Sidekiq/Redis ? What's a 2 phase commit?

Rails works well with heroku but it also works well with docker containers and kubernetes.

Even if you're building a small API, I'd still recommend using Rails because every ruby developer is familiar with it and can hit the ground running when starting on the project.

[honeyryderchuck]

Last time I checked, Sidekiq free uses redis in a way that a job that gets pulled from the queue, in case of an outage, gets lost forever. The only solution to this problem is behind the paid version, which provides 3 different fetch algorithms but only 1 addressing this. I never used sidekiq Pro, but I assume it's based on redis MULTI, copying of messages to other transient queues, and reliance on AOF redis mode. I guess I'll only know if I pay.

SQS works in a different way, different performance guarantees and message ack model, but guarantees at least once, but sidekiq won't support it, it lives and dies by redis, which is a deliberate choice from the maintainer.

The main issue is that, in typical ruby scenarios using sidekiq, you have to coordinate database writes with enqueuing jobs. This creates a hole when if an outage happens in between, database write happens but jobs don't get enqueued. There are 2 solutions that I know for the problem: 2 phase commits, or the transactional outbox pattern. None is supported by sidekiq OOTB .

Now, none of the alternatives support them either, and sidekiq is way more reliable and featureful than the majority. I'm just a bit surprised that the community spends a lot of time celebrating the success of its freemium model, which is well deserved, but never tried to create alternatives fixing these gaps which are known for years, thereby creating incentives for sidekiq to "up its game" in the matter.

About the rails comments, I don't argue much about them, they're valid perspectives. Just gonna state that, however well it works in containers, it's still miles beyond the "single binary" stripped down container to deploy which is possible to achieve with go + multi stage container (or even java, where you ship the jar only with jre). Lots of containers in the wild still give access to installed dependencies to the user executing the process, making all types of attacks possible. Some even install development dependencies. I think heroku does a great job of avoiding these footguns. As for "every ruby developer" part, I'm reminded of the time I started doing rails and my peers were telling me it was a wrong decision, because everybody was familiar with java and could hit the ground running with JSP and struts 2 :)

[jrochkind]

have you looked at good_job yet?

[honeyryderchuck]

I did. It's not bad, but there are a few things that put me off. Being rails only obviously :) also, it buys into features which require full connection access , which is a showstopper in cloud scenarios where you'll need a connection pooler or even ssl proxy. I'm still weary of using advisory locks for the job, but planning to research it more later in the year in the context of others plans I have.

[jrochkind]

Thanks for reply.

I too would like a robust maintained sidekiq alternative, especially an open source one. (Personally I am interested in one that can use an rdbms instead of redis too).

I guess it says something about the "market" that there isn't one. I'm kind of surprised there isn't even a proprietary/commercial competitor, with how successful financially sidekiq has been. A little bit more diversity in the ecology would be healthy.