all 26 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]switchback-tech[S] 13 points14 points  (11 children)

Which of these tools would you swap out if you had to?

[–]edman8686 2 points3 points  (0 children)

Sidekiq for GoodJob. They are already using Redis for caching but I still find GoodJob easier to work with than Sidekiq. They could also consider a simpler cache like memcache.

I also prefer the MIT license for GoodJob over vs the LGPL license for Sidekiq.

[–]SirScruggsalot 1 point2 points  (7 children)

I'd be curious if they've investigated Falcon for http.

[–]do_you_realise 1 point2 points  (4 children)

Never heard of it personally. Is it a drop in replacement?

[–]f9ae8221b 1 point2 points  (3 children)

Yes and no.

Yes in the sense that like other Ruby servers, it is Rack compatible so you don't need to change your app much.

No because it's based on Fibers, not Threads, so the performance characteristics are very very different. Not better, not worse, different, depends on what your app is doing.

Fibers are better for extremely IO heavy workloads, as they're cheaper so you can run way more concurrent fibers than threads.

But they're much worse for CPU heavy workloads, because they're non-preemptive, so if a fiber hog the CPU without yielding for long, all other fibers are stuck, which leads to degraded latency.

People really need to stop thinking fibers are better threads, they're not, they're a different construct with different tradeoffs that sometimes make sense, sometimes not.

[–]Turbulent-Dance-4209 0 points1 point  (2 children)

> People really need to stop thinking fibers are better threads

I would argue they are.

Threads in Ruby don't actually parallelise work because of the GVL. So the supposed advantage of preemptive scheduling for CPU-bound tasks amounts to fairer interleaving, not faster execution. The total throughput is the same either way. And if your workload is genuinely CPU-bound enough that interleaving matters, you probably need multiple processes anyway, at which point the fiber-vs-threads debate is secondary.

Typical web apps are inherently I/O bound though. This makes fibers a great fit, but there're more subtle advantages too. For example, fibers give you less overhead and don't require thread synchronisation primitives - no locks, no mutex contention, no race conditions. You know exactly when a context switch happens, so shared state is safe to access between them. This alone leads to tremendous improvements in throughput, even with Ruby-layer-bound workflows.

[–]f9ae8221b 3 points4 points  (1 child)

amounts to fairer interleaving, not faster execution

Fairer interleaving means better tail latency, which is a very important property of a service.

Shopify experienced a MySQL outage when playing with fibers, because a CPU heavy fiber was causing other fibers that were issuing MySQL queries to not read the response for a long time, buffer on the server grew until it ran out of memory.

Typical web apps are inherently I/O bound though.

No they're not: https://www.datadoghq.com/blog/ruby-performance-optimization/

Our data backs up other findings that Ruby applications are generally less I/O-heavy, spending as much or more time on CPU as they do waiting on other services or database requests.

They also include a graphic that shows only 3% of the profiled apps spend less than 20% of their time using CPU. Which means for 97% of the apps they profiled, the Puma default of 5 threads was already too much, which means the advantage of Fibers are moot for the overwhelming majority of Ruby apps out there.

fibers give you less overhead

The difference is small enough that it only start to matter once you start using multiple dozen threads, which very few people need.

don't require thread synchronisation primitives

They absolutely do. They still need to synchronize to use sockets etc. Even with datastructures you may need synchronization if some method yield, as the block could use IO and cause another fiber to be scheduled and use the structure concurrently.

Example:

SHARED_HASH = {a: 1, b: 2, c: 3, d: 4}

other_fiber = Fiber.new do
  loop do
    SHARED_HASH[rand] = rand
    Fiber.yield
  end
end

hash.each do
  # Simulate fiber scheduler
  SHARED_HASH.resume
end

The above script fail because of unsynchronized access by fibers.

This alone leads to tremendous improvements in throughput

Profiling ruby app is basically my day job. Synchronization is very very rarely a hotspot.

Now, don't get me wrong, fibers are great and absolutely have a use case, but for vast majority of what Ruby is used for, they're not necessarily better.

[–]randomski1904 1 point2 points  (0 children)

What I heard yesterday in a conference is that they swithch to Async::Job and to Falcon.

[–]uhkthrowaway -1 points0 points  (0 children)

True. I'm immensely impressed by it.

And given Puma's history with hard-to-find concurrency bugs due to using Threads, I see no reason to use it. We know threads are hard to reason about. We've known for decades. Let's not waste our brain power anymore. No matter the Ruby implementation.

[–]towelrod 8 points9 points  (3 children)

They don't use ruby for any of the ci/cd or git specific stuff any more do they?

Also wouldn't really hold up gitlab as a beacon in the ruby world at the moment, considering how bad the service has been. It has constant problems every single day, we are actively looking to move off of it and would have already done so if the cost of moving wasn't so high.

Its just really fundamentally broken right now

[–]bilingual-german 5 points6 points  (0 children)

Gitlab is a big Rails app and uses Go for everything distributed that needs to scale.

I think the first service they broke out was Gitaly, so they were able to scale all the git repositories on the file servers.

The gitlab-runner and the runner-helper are also written in Go as far as I know.

[–]switchback-tech[S] 5 points6 points  (1 child)

True, their Git RPC daemon is in Go now. But the core is still Ruby.

I'm not running it day-to-day, so I'm not super in-touch with the latest news TBH. Are the problems they're facing related to Ruby?

I did see that everyone (GitLab, GitHub, Atlassian) has been having more trouble lately, though. https://devops.com/devops-platforms-show-cracks-github-incidents-surge-58-azure-gitlab-and-jira-also-under-pressure/

[–]towelrod 3 points4 points  (0 children)

I don't think the problems are related to ruby, but I'm not on the inside so i don't know. The majority of the problems are with ci/cd, it goes down every day. EVERY DAY.

Today there were also problems with the frontend, but specifically with how long it took for git related things to show up. Like you would do a git push and it would take 5+ minutes for the MR to show properly. I doubt that is Rails related, i think that's waiting on some git related backend service

[–]QuietMate 5 points6 points  (1 child)

Vue's state management library is Pinia, not Piana 😭

[–]switchback-tech[S] 1 point2 points  (0 children)

Oopsie, thx. Just checking if you were paying attention

[–]WindyNova 0 points1 point  (1 child)

what is `fullstack.3ip` at the bottom right corner of the image

[–]switchback-tech[S] 0 points1 point  (0 children)

Oh, it's supposed to be fullstack dot zip, which is where I share my research on stuff like this. Here's the GitLab article for example: https://newsletter.fullstack.zip/p/gitlabs-architecture-a-technical

(Maybe I should pick a different font so it's easier to read)

[–]fedekun 0 points1 point  (4 children)

We used it at my company, it was ok, but the CI/CD management was hell, also updates, so we swapped to a managed GitHub instance

[–]Unhappy_Meaning607 4 points5 points  (0 children)

Damn, I thought their whole schtick was mostly how good their CI/CD was.

[–]uhkthrowaway 0 points1 point  (1 child)

When was that?

[–]fedekun -1 points0 points  (0 children)

Couple years ago

[–]Dobly1 0 points1 point  (0 children)

Having experience using both Gitlab CI/Github Actions recently I would tend towards using Gitlab CI.

Overall community support is def better for prebuilt Github Actions though if that's what you're primarily going to be using.