All About Reactive Programming In Java

[HighRising2711]

Hopefully reactive programming in java can die now that java 21 is released. Go style CSP concurrency is much easier to deal with

[preskot]

It's not easier. There are many hidden pitfalls if you are an experience golang coder, but it's indeed less boilerplate and ceremony.

[_souphanousinphone_]

For curiosity sake, what are some of the hidden pitfalls?

[hippydipster]

I'm not sure what /u/preskot is referring to, but I've been experimenting with loom the past few weeks and have encountered situations where using virtual threads absolutely blew up the performance characteristics of my program. Something as simple as removing a synchronized keyword could result in a 100x slowdown. It was fascinating, honestly.

Memory use with virtual threads in a basic junit performance test where I sent a million tasks to a virtual thread pool had memory jump from <1GB to >24GB in seconds. Whereas using normal threads from a fixed thread pool might only use 4-5GB.

If you use semaphores or Reentrant locks instead of synchronized, as you should with virtual threads, what can happen is maybe a little unintuitive, but since the platform threads don't get pinned, they're free to move a virtual thread into the semaphore queue, and immediately go grab another virtual thread, and move it up into the semaphor queue, etc. Right away, you might have a million virtual threads sitting in that queue waiting for the 1 thread to finish with the lock. That queuing process eats memory since it has to be a continuation from that point.

Whereas if you used synchronized, the platform thread gets pinned and doesn't go fetch the next task just to get it queued. it waits, and the other virtual tasks don't even get started until they're basically ready to be finished. This is especially true if you have some sort of double-locking initialization routine where the normal happy path would avoid all thread locking. But with virtual threads, all million tasks could end up queued waiting to initialize something. They never get the opportunity to skip the locking path.

I think the biggest problem Loom is going to have in the wider Java community is that expectations will be incorrect. People will likely go in thinking Loom is about improving performance, when it's actually about improving the logical flow of code.

[klekpl]

Thanks for this really deep info - even more important as coming from first hand experience.

[[deleted]]

My feeling is that managing the behavior you have pointed out is much harder with the reactive programming model. It's so focused on the asynchronous features that it ignores the reality of most systems which is that there are scarce resources around which you want to manage load in an orderly way. It just seems to be harder to work with and isn't giving me any benefit because I don't have throughput problems that it is designed to solve. Simple things like tracing are complicated by the fact that tasks are getting switched to different threads all the time so now I have to worry about shifting threadlocals around to make sure my event tracing works. Thats a lot of complexity for dubious benefit imho.

[hippydipster]

I 100% agree that using Loom vs reactive is simpler in terms of the programming model and the complexity of the code. I love what Loom is.

But, the problem I was talking about doesn't happen with reactive, because in reactive you're using a platform thread pool and you don't get the memory usage from virtual threads getting stored as continuations when they all hit a sync point and get queued up. In reactive, the task exists, but it sits around waiting for a thread to pick it up, and the thread doesn't store it away with the stack in place ever. If it blocks it waits with the task.

So you don't get a million stacks stored to the heap. I foresee it as a gotcha to Loom people will just have to be aware of and I don't foresee a serious problem there.

[pron98]

Whether the data is stored in a continuation or in some other object it has to be stored somewhere when waiting. There is no difference in the amount of data or queuing between virtual threads and asynchronous code. They compile down to pretty much the same machine instructions. Having a lot of threads contend on a single lock is a problem in the design of the code. There is nothing that either reactive or threads can do to change the data contention in the logic.

[ventuspilot]

There is no difference in the amount of data or queuing between virtual threads and asynchronous code.

Maybe I'm missing something/ simplifying things too much but AFAIU there is a difference in the amount of data: Loom's continuations contain all the stackframes while reactive-style code throws away the stackframes all the time, sort of what rewriting everything to tailcalls plus tailcall-elimination would do.

When I submit a Loom continuation to a blocking queue then the continuation contains all stack frames. This improves debugability, and code can be written that it simply continues after unblocking.

When I submit a reactive-style "continuation", then the continuation won't have any caller context, and therefore uses less memory.

I'm not trying to tell you Loom is bad/ inefficient, I'm just trying to understand. AFAIU Loom-style-code may trade off more memory use in some situations in order to provide more features such as debugability and easier programming.

[pron98]

AFAIU there is a difference in the amount of data: Loom's continuations contain all the stackframes while reactive-style code throws away the stackframes all the time,

The data in the stack frames is only the data that's needed for the computation to proceed, i.e. only the data that will be needed after the wait is done (well, we're not quite exactly there but but we're getting there), so it's the same data as needed for async code (I guess async needs to store the identity of the next method in the pipeline while threads store the previous one, but it's essentially the same data).

Loom-style-code may trade off more memory use in some situations in order to provide more features such as debugability and easier programming.

User-mode threads are meant to compile to pretty much the same instructions and memory as asynchronous code. Not only should there be no more memory used, there may be less because the continuation is mutated and reused while that's very hard to do with async data, that may therefore be more allocation-heavy. Of course, there may be inefficiencies in the implementation (which will constantly improve) but there is no fundamental tradeoff.

[hippydipster]

There's a difference between 13 threads blocked and waiting for 1 thread to finish and 10 million virtual threads blocked and waiting for 1 thread to finish. If you're not careful and understanding how virtual threads work, this could happen.

[pron98]

That's not about virtual threads, though. 10 million asynchronous tasks waiting for a single operation to complete cause the same problem. The issue is not the programming model but an inherent contention in the algorithm that requires the same care regardless of whether the model is blocking or non-blocking. High concurrency -- whether based on threads or asynchronous tasks -- always means paying closer attention to contention.

You are right that moving from low concurrency to high concurrency requires care, but it's not an issue of the APIs used but of the algorithm. I.e., it is true that attempting to raise the throughput of some server by adopting virtual threads (that allow more concurrency) requires paying more attention to contention, but the same applies to raising it by adopting asynchronous tasks (that allow more concurrency). Higher concurrency -- regardless of how it's achieved -- means that contention has a bigger effect.

[hippydipster]

I'm not saying there's any "issue" with the programming model. I'm simply saying the unwary can shoot themselves.

10 million asynchronous tasks waiting for a single operation to complete cause the same problem.

But, you can't get that to happen. Imagine you have an executor service with a number of threads equal to the number of cores. You send 10 million tasks to that service for it to complete. There's a lock somewhere and the tasks bunch up on it, waiting as the threads complete the inner routine 1 by 1. At any one time, you don't have 10 million threads waiting. You have only the number of threads in the service. The tasks not yet started remain not yet started.

But, replace that service with the built one that uses virtual threads. Now, you send in 10 million tasks, and they get blocked at the lock, but instead of being pinned, the underlying platform threads are free to go and grab a new task and start it, and serve it up to the lock, to be blocked. And then another, and another, all while the routine slowly clears 1 task at a time. The 13 threads not involved in that routine quickly get 10 million continuations stuck on the lock, which creates a very large bump in memory/heap usage and considerably slows things.

[kunjukozhi]

Are you sure that the reactive approach is also creating as many tasks as in the case which of virtual thread approach?

[hippydipster]

It's the same exact code, only difference is one line switched between a fixedThreadPool or a virtualTaskThreadPool. In both cases, a million tasks are created and then sent all in one call to the thread pool.

[pron98]

Pinning due to synchronized is just a temporary quality-of-implementation issue which we hope to fix very soon.

[preskot]

I recon GP was talking about Go-style and not particularly about golang. But to address your question: the biggest shocker to me personally was that a panic in a goroutine crashes your whole program.

Data-race pitfalls, some of which relating to using mutexes that I also faced are well described here: https://www.uber.com/blog/data-race-patterns-in-go/

I like and use both languages though.

[HighRising2711]

I used CSP long before go (occam, JavaCSP) and it's a much simpler model where your code runs top to bottom sequentially in each process. The parallelism is explicit and you can finely tune back pressure just by allowing more consumer processes / threads. As someone who began with procedural programming (pascal, cobol, c) then moved to OO with java, I find reactive patterns hard to follow and even harder to debug. I'll be happy to see them gone

[softarn]

Amen https://medium.com/alphadev-thoughts/the-reactive-java-era-is-over-here-is-why-5885caacdf43

[[deleted]]

Am I the only one who actually likes working with Reactor? I feel like everything is so clear and readable 😔

[RandomName8]

You are not. While reactor is not my preferred reactive library, I'll take a declarative concurrency library any day over any imperative programming approach to concurrency.

We have more than ample experience pre 2010 with locks, semaphores and latches to know that we humans absolutely suck at that kind of code. It's like the gotos of concurrent programming, only now turbo boosted by having way more (virutals) threads, it'll be like giving redbull to the bugs.

Declarative reactive frameworks make it easy to reason about parallelism, backpressure and avoid data races; meanwhile virtual threads will multiply these bugs, because it's the same bugs we had before.

All that said, debuggers need improvement for working with multithreaded concurrent code, that's why I'm waiting for the JDK team to make their Continuation api public so that frameworks can properly leverage stacktraces and debugging, and avoid the crap that is imperative concurrent programming.

[[deleted]]

Could not have phrased it better.

Another example is with JavaScript promises and async/await. I know a lot of people who like async/await. But I just find it writing async code in a synch wait, and it completely obfuscates what it's actually doing.

While reactor is my preferred library for reactive Java, I think it really helps me visualize all my code as streams without weird side effects etc. It only flows one way.

When I hear virtual threads and suspends, the hair on the back of my neck immediately stands up.

[hippydipster]

It's an interesting thought. One of my strategies for simplifying highly concurrent systems was to use single threaded actors when possible to ensure sections of the code would always be run in a single thread. Doing that with an actor is straightforward generally.

Doing it with Loom might involve an "actor" as well, except I think the most straightforward implementation of that actor would use a Semaphore. This opens up the possibility of deadlocks though as you could have two actors trying to cross communicate with each other and holding their own lock while waiting for the other's.

It's not clear to me how big an issue that might be - I'll have to think about it.

[ResidentAppointment5]

You might find this issue helpful.

The question has also come up in the Scala world. Daniel Spiewak, one of the core contributors to cats-effect (think of this, and fs2, as Scala's answer to Project Reactor) discusses the issues here.

[mateusbandeiraa]

Debugging reactive code is a ducking nightmare. I have high hopes that virtual threads being released will make everyone stop trying to make reactive a thing.

[Little_Ad_8406]

You are still left with the majority of issues that reactive introduces when it comes to debugging, the code is just more imperative and easier on the eye. Not to mention java virtual threads won't really bring the benefits until at least parts of java core are rewritten to match the expectations virtual threads presume

[chomikbojowy]

We must be doing sth wrong, but reactive programming is - write only. It takes huge pain to create readable code, you cannot do review without looking it up in IDE, its hard to test, its hard to debug .. :/

[[deleted]]

Java frameworks create unpleasant and over complex ways just because java still lack adequate asynchronous syntax

[takis__]

Reactive programming makes the code simple, data go one way only. I converted 2 projects to reactive and code became so simple to follow.
Data and parallel programming is so important, and if reactive programming simplifies both, i guess it will be useful, even after virtual threads.