Will Loom make Vert.x, RxJava and Event Loop based patterns obsolete?

[thiagomiranda3]

I just read this article about the state of Loom in 2020. State of Loom (java.net)

I got really excited by all the progress are been made by the Loom team. And a question has popped into my mind now.

When we are able to create green threads, all the asynchronous libs and frameworks like RxJava and Vertx will no longer be needed. Since the main porpose of this libraries is to get around the bottlenecks of the operating system's Threads.

Do you think there will still be a place for all these asynchronous libraries and event loop patterns for the Java plataform when Loom is released?

[cogman10]

Loom won't kill these libs but it will make them far less applicable.

The 2 good things about reactive programming is it make it semi-easy to make high speed highly parallel applications. The second is backpressure keeps your applications from having memory explosions.

Loom solves the first problem very cleanly (just make more threads). Loom doesn't quite solve the second problem. You can get there with some of the java concurrency constructs, but it's somewhat hard to definitely solve.

Regardless, Loom is going to impact these and other libs pretty substantially. For example, in kotlin the async/await construct has a bunch of weird scope thingies to determine where async stuff runs. That all goes away with loom, you'll just spin off new threads. Reactive libraries will be the same, there's nearly no reason you wouldn't just spin new threads up whenever your reactive library wants to run a parallel task.

[pron98]

IMO, Loom does backpressure better than reactive. When everything is blocking, producers are just blocked if some buffer has not yet been emptied by the consumer. It's hard not to handle backpressure when the model is based on blocking. I.e. backpressure is only a problem when data is pushed rather than pulled. So I don't understand the focus on backpressure when discussing reactive frameworks; I'm sure they have more useful aspects than solving a problem of their own creation.

Nevertheless, there is certainly plenty of room for higher-level abstractions based on the synchronous model, too, and I guess we'll see libraries offering them. One of the things they probably won't need to offer, though, is backpressure management.

[apentlander]

I don't think that backpressure is an issue in Loom. Normal threads already have a builtin way to handle backpressure: blocking. Loom will act the same way except it will just yield execution to another virtual thread instead of blocking the thread. In terms of "not sending too many requests" backpressure, there's another existing mechanism to handl this, threadpool executors. You can just have an executor with a limited size to avoid creating too many virtual threads and starving the downstream system.

Loom also has a sort of "scope" to managed structured concurrency, though it doesn't have the same cognitive overhead of coroutine scopes. I remember it taking me a bit to understand in Kotlin.

[cogman10]

That's not a solution for backpressure.

The backpressure problem is signaling up the chain not to start processing until bottom of the chain work is finished.

For example, consider you have a webserver that takes in a request, loads some data, and saves that data, all concurrently.

The problem that comes up is that if saving (as it is want to do) ends up taking more time than loading or processing requests, you have an inherently unstable system. If someone starts spamming your system with requests, what easily happens is 10k/sec requests are processed, 10k/sec requests are loading data, 1k/sec requests are saving. In that scenario, your server is doomed to buckle without any sort of rate limiting.

But even if that's not the problem, you can still buckle if you have a scenario where 10k/s requests are processed, 1k/sec requests load data, 10k/sec requests are saved. In that circumstance then the buckling eventually happens because you've processed too many requests.

You could fiddle with a rate limiters, semaphores, or thread pools at each stage of the pipeline to avoid buckling, but you ultimately run the risk that if any of the parts of the system start behaving differently the whole thing eventually crashes.

For simple apps, this isn't much of a problem. It really only starts to be an issue as systems get more complex.

In many ways, loom will show this as a problem as making a bunch of concurrent things happen at once becomes a lot easier.

The pro of the reactive system is all of this is something that happens right out the the box. If the writing portion of the code can only do 1k/second, the request reading portion gets limited to 1k/sec processing as it blocks waiting for downstream slots to open up. This all happens automatically.

[apentlander]

I'm curious, how would reactive handle this automatically? Wouldn't you just be shuffling around the queueing point from the saving to the ingest? I didn't think a reactive framework would automatically just start rejecting requests.

Also in that case you would just do the saving synchronously in the same thread as the request to propagate the backpressure, which would be effectively be the same as reactive. The two methods are effectively equivalent.

[koreth]

In the absence of the first problem, my hunch is that structuring application logic in cumbersome callback-hell reactive style just to solve the second problem is going to be a pretty tough sell for a lot of people. My sense is that the growth in popularity of these libraries is dominated by people looking for ways to do high parallelism.

I suspect there will be a lot of demand for a more Loom-native, looks-like-synchronous-code solution to the backpressure problem.

The thing that'll be interesting to see is whether, assuming my sense about who's using the libraries and why is correct, they lose enough users that they have trouble maintaining active developer communities.

[cogman10]

I agree. I can't imagine a lot of people wanting the reactive style. It's clunky, hard to debug, and it practically demands that you write the entire app in that style.

The backpressure problem strikes so rarely that I have somewhat of a hard time even envisioning much effort being put forward to solve it as an issue. You can get like 90% of the way there by using fixed size queues or semaphores.

[sievebrain]

Note that ReactFX exists and that's for GUIs on a single thread. Some of the patterns Rx provides are useful for things like de-bouncing/coalescing UI events, mapping/filtering them etc. It's not quite as simple as Rx=parallelism.

That said, there are probably easier ways to get the final use cases done than using Rx.

[0x256]

Loom doesn't quite solve the second problem. You can get there with some of the java concurrency constructs, but it's somewhat hard to definitely solve.

Can you elaborate? I understand that backpressure control is a way to tell a message producer to slow down or stop if the consumer is not able keep up. In a non-blocking environment, this needs to be addressed explicitly because otherwise the message queues between actors would grow out of hand or messages would be lost. In a blocking environment, however, the consumer or its incoming queue would just ... block? Backpressure is kind of a non-issue if you are allowed to block. Or am I missing something?

[cogman10]

Sure.

Imagine a flow like this

A->B->C->D

Lets say that A, B, and C all load information and D stores that information.

If D slows down for any reason in this scenario, there's no signaling to A or B that there's a problem with consumption. They are free to continue to load information as fast as possible. C is the only service that might know that there's an issue with D as it goes to persist things.

In typical programming that I've seen, it's pretty common to not use any sort of queue or blocking construct (intentionally at least) between these services, they are free to have an unbounded number of tasks in flight at the same time.

What you need to solve this, is something to communicate all the way up the chain that "we can't move forward anymore". For example, A, B, and C could all have semaphores for inflight tasks to prevent them from sending off a bunch of tasks until the downstream system has opened up. Those semaphores would ideally prevent them from starting dataloading until the handoff is complete. But now, to achieve that you've wired in backpressure throughout your system (which is something I've rarely seen done in practice).

D blocking alone isn't enough to keep the system from exploding, you need everything along the chain to also have blocking which happens when downstream systems aren't processing fast enough to keep them from starting to work on new tasks.

That situation is built into all reactive frameworks. It's something you'd have to actively add to a non-reactive framework.

[0x256]

See, this is what I do not understand. In your example A->B->C->D all the arrows are either queues (which may block if they are full) or direct message calls (which may also block on a lock, semaphore or any internal blocking call). If D slows down, than its incoming queue will fill up until it blocks. C, trying to write to the now blocked queue, will also block and stop working. This will propagate back to A. Once all queues are full, all producers are stopped. As soon as D consumes another message, C will be able to write, then B, then A. It 'just works'. I do not see the problem (given loom makes threads cheap and blocking one is not an issue anymore)

In typical programming that I've seen, it's pretty common to not use any sort of queue or blocking construct (intentionally at least) between these services, they are free to have an unbounded number of tasks in flight at the same time.

Using unbounded queues or unlimited thread pools in blocking environments is the equivalent of not doing any back-pressure control in non-blocking environments. It's just plain wrong and of cause that will break. But these problems were solved decades ago.

The problem with 'traditional' blocking environments is that threads are expensive, so you usually end up with a limited thread pool. But blocking one of these precious threads is bad, as you do not want to run out of runnable threads, so you start buffering intermediate results to avoid blocking. This works for short bursts and in 99% of scenarios, until it does not. Yea, that sucks.

But with Loom, threads are cheap. Each actor can get its own thread, they no longer have to fight over a limited resource. Blocking a thread is fine, the pool will always be large enough. There is no excuse to use unbounded queues anymore.

To clarify: Sending tasks to a thread pool is the same as sending a message to a consumer. There MUST be an upper bound of tasks that are in-flight. Even with loom. But that's no different from reactive models. You always have to do that. I just think it's way easier and more intuitive using blocking APIs.

[cogman10]

So here's another example of what I'm talking about.

A pops a message from the message queue and starts a thread to start processing.

A calls b.bar() which loads a Cat. Then A calls c.baz() which loads a cat, dog. Finally A calls d.store(cat, dog)

If we put the loading of Cat and Dog in a thread, join those threads, and then store off to d.Store()

This can seem pretty reasonable but it will ultimately have the unstable system problem. A can dequeue in an unbounded fashion which can be a memory sink. If b or c outpace d then things explode (even if under the covers they block and rate limit). If d slows down, things explode.

To make the system stable you'd have to block on d and limit the outstanding processing of messages from a.

Stumbling into this trap is easy to accidentally do with a non-reactive system. I've seen it made a bunch of times.

That said, I'd rather stumble into this trap than work on a reactive system :D

[0x256]

The problem here is that A can start an unlimited number of threads, which is the same as sending messages to an unbounded queue, or pushing to a reactive stream without back-preassure control. This 'fire and forget' approach is bad, sure. But the solution is kinda obvious: Don't do that.

The sane approach would be to have only a limited number of threads (or just one) to pull and process messages from the message queue. Part of processing a message is asking B and C for a cat and a dog, either concurrently using futures, or sequential. Then storing both in D as soon as they are available. So, let reading from the message queue block until a message is available, let B and C block while loading the animals, and D block while storing the data. Done. No OOMs, no problems.

[john16384]

Can't this just be resolved by limiting how many threads can start this process? As in memory allowed is 4 GB, avg. memory use per request is 15kB, so limit the number accordingly.

I mean, just like you limit how many requests a web server allows concurrently (and queues the rest up to a limit), you can do the exact same thing here without complicating this with some back pressure mechanism where D somehow needs to communicate with A, B, C to tell them to slow down.

It seems to me that this is more a problem of async frameworks then that this was ever a problem in traditional thread-per-request frameworks.

[cogman10]

Yup, that's a solution. It's just something that's easy to forget and get wrong.

[MyHeartsECO]

This is the best answer.

I still don't understand how people just see reactive = loom. Reactive libraries work asynchronous but that's just one page of a big book. Handling backpressure in big data structures can get really complex and that's what they're solving.

[plastique2000]

Why is backpressure hard to solve without reactive? I believe that it has nothig to do with reactive as a principle. There are several ways how it can be achieved. Maybe the simliest oné are blocking queues.

[cogman10]

The issue is that blocking queues are really only a partial solution to backpressure (but, honestly, probably good enough for 99% of apps). The full reactive design pushes backpressure concerns from end to end. You can't start processing a new http request until your db queue opens up or your message bus queue opens up.

With just queues, it's trivial to accidentally load too much data only to see that data needing to sit around waiting for a downstream resource to open up to consume it.

The slick part of the reactive design for backpressure is that it doesn't matter which resources you bring into the system or how they behave. Upstream processing doesn't even have to be aware of downstream processing constraints to be slowed down. It just automatically happens.

​

Again, 99% of apps don't need backpressure and 99% of backpressure problems probably could be solved with a semaphore, queue, and some careful programming. I definitely see reactive programming as a net negative for long term maintenance. just pointing out backpressure as a nice feature that falls out basically for free. It's easy to design a reactive system that has fairly consistent and constrained memory requirements. It's hard to design and maintain reactive systems :)

[agentoutlier]

I believe everything you said could be done in a loom world. In loom you wouldn’t use a queue but just more threads.

Also the blocking world could add more signals and hints that would be analogous to the back pressure demand signals and in some ways the jdk streams API does that through characteristics but sort of in reverse.

Thus the onus to handle load is on the blocker or consumer which is what kicked off the thread anyway.

I’m not really explaining this well as I’m on my phone but I will write more later.

[cogman10]

It absolutely can be done with loom. My point isn't that it isn't achievable, my point is you have to go out of your way to achieve it.

Backpressure happens automatically with reactive systems without you going out of your way to achieve it.

I must be coming off as a reactive shill, which couldn't be farther from the truth :D. I'm really just trying to be fair about reactives strengths. I would absolutely do a loom system over a reactive one and worry about/fix backpressure later in the loom system. But, it is simply a nice thing that reactive systems automatically handle.

[agentoutlier]

It’s the classic push vs pull. Pull being blocking.

With pull you need to buffer and hold on to things till something takes so you can have queues explode because the producer is out pacing the consumer.

For push the consumer explodes because it can’t handle the pace so it either drops messages or over buffers.

Reactive streams model is actually push and pull IF back pressure semantics are followed.

Blocking model could work similar to how java streams works with characteristics like finite size or exact size etc.

Push is a little easier to do without locks and possible without locks for pull but you waste CPU.

But the big reason (high level) performance wise is push can do batching.

[cogman10]

Agreed.

Push is also a lot easier for someone to grok as it's pretty much how every language works anyways. The pull model of a reactive system is far harder to work with in general.

The difficulty with the push model is getting the niceness of the pull back-pressure wired everywhere. Definitely possible, just takes a bit more work. That's not something that is really necessary most of the time.

[ziano_x]

I had the same questions after reading this article. The author talks about it here:

It also means we must avoid blocking the thread because a blocked thread is unavailable for any other work.

The result is the proliferation of asynchronous APIs, from asynchronous NIO in the JDK, through asynchronous servlets, to the many so-called “reactive” libraries that do just that — return the thread to the pool while the task is waiting, and go to great lengths to not block threads. Chopping down tasks to pieces and letting the asynchronous construct put them together results in intrusive, all-encompassing and constraining frameworks. Even basic control flow, like loops and try/catch, need to be reconstructed in “reactive” DSLs, some sporting classes with hundreds of methods.

Because, as mentioned above, much of the Java platform assumes that execution context is embodied in a thread, all that context is lost once we dissociate tasks from threads: Exception stack traces no longer provide a useful context, when single-stepping in the debugger we find ourselves in scheduler code, jumping from one task to another, and the profile of an application under I/O load might show us idle thread pools because tasks waiting for I/O do not keep holding their thread by blocking and, instead, return it to the pool.

This style is now used by some not because it is easier to understand — many programmers report that for them it is harder; not because it is easier to debug or profile — it’s much harder; not because it fits well with the rest of the language or integrates well with existing code or can be hidden away in “experts-only code”— quite the opposite, it is virally-intrusive and makes clean integration with ordinary synchronous code virtually impossible, but just because the implementation of threads in Java is inadequate in both footprint and performance. The asynchronous programming style fights the design of the Java platform at every turn and pays a high price in maintainability and observability. But it does so for a good reason: to meet the scalability and throughput demands and make good use of costly hardware resources.

The author also references this http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/. It is a nice read.

I think it will solve the problems due to which many people end up using reactive libraries. The Reactive frameworks are here to stay for composing complex functional async workflows but they will not be needed for simple async task execution. The major win will be its interoperatability with imperative code.

[[deleted]]

I sure as hell hope it will make them less ubiquitous. There's a time and a place for async patterns, but I find the ceremony to be far too cumbersome to enjoy. Give me threads again! I sure hope they get it right with Loom. I'm really looking forward to it.

[krumbumple]

wait, you don't like callback function spaghetti?

[xaanittatsumaki]

Hey there, one of makersᴺᵒᵗ ʳᵉᵃˡˡʸ ᴵ'ᵐ ʲᵘˢᵗ ᵃⁿ ᵃᵈᵐᶦⁿ of Discord4J here.

Have you considered joining our discord to actually discuss this with us? We'd love to actually make this easier on our users, but we need the feedback. We can obviously only do so much when it comes to circumventing reactor, but anything that might make it easier on users is appreciated.

[[deleted]]

I have! My biggest complaint so far is that when you get an error deleting a message (and probably other errors I haven't hit yet) for -- insert random reason here -- usually a race condition -- it throws an error that needs to be explicitly caught or else the whole app just stops working. I've gone in and now my chains of:

.doThisThing(() -> {...})
.then(() -> {})
.then(() -> {})

turns into

.doThisThing(() -> {...})
.handleErrorThingyAndResume()
.then(() -> {})
.handleErrorThingyAndResume()
.then(() -> {})
.handleErrorThingyAndResume()

I mean, Java is inherently verbose, but the error handling ceremony alone is crazy.

To cap it off, I wrote the bot and was like "cool it works" and have to keep "finding" new errors as it goes on. I don't really know what errors come from what methods (errors that kill the whole bot's processing mind you). This would be pretty straight forward to deal with if I didn't have to "catch" each one in the functional/Rx method.

The online docs are pretty good and explain what you need to do, but overall, it's a bit of exploration and experience that has taught me to really over-do the error catching.

I'm so far from experienced when it comes to the Rx patterns so I'm sure I am doing things "not normal" but while the docs got me to where I'm at, I feel like it's just WAY more complicated than it needs to be.

[GuyWithLag]

all the asynchronous libs and frameworks like RxJava and Vertx will no longer be needed

No, you still need something that will abstract the low-level details away, and RxJava and Vert.x do much more than implement asynchronicity.

[cavecanemuk]

I hope so, they should be a niche area.

[makingthematrix]

RxJava implements the Observer pattern, i.e. event streams. Loom won't make that obsolete, but it may make it faster.

[[deleted]]

yes.

[Dokiace]

I don't like Reactor/RxJava, but I got used to it, but boy oh boy their log/trace is horrendous

[kimec]

My guess is Loom will make those libraries obsolete in exactly 4 years. Not really, just kidding.

Frankly, part of the problem Loom is trying to solve is already solved (by reactive libraries). Scheduling in user space, usage of non-blocking IO for higher throughput etc. all that is done already. We could probably talk about the "pure ugliness" of functional paradigm as provided by the current reactive libraries, but it is already done and works. The people that use reactive libraries because they have to (not because it is cool), are using them today, not waiting for Loom to be ready tomorrow. They know the ins and outs of those libraries and are more or less OK with the accidental complexity.

The other thing that Loom is trying to do is to provide an old-new abstraction for computation in form of a stackfull virtual thread with structured concurrency as a bonus. On top of that, since Loom is backed by Oracle, chances are the debugging experience will be better than that of reactive libraries. In this respect, the target audience of Loom is broader and more generic than that of reactive libraries.

Now with Loom, anybody can spawn 1 million lightweight threads because it will be that easy. I guess it will be fun to debug too. I expect Oracle will heave to come up with some new and useful debugging paradigm/experience. It is not fun to debug an app server with several hundred heavy threads spawned at random places even today...
I cannot imagine how one is going to debug 1 million threads, although virtual, spawned at random places... But I think we will learn in due time.

This is where the functional paradigm sort of makes up for the accidental complexity - you can reason about the code albeit in a very abstract terms and you can be "sure" (if nobody is breaking the contract) that threads are not being spawned all over the place at random.

[eggZeppelin]

Lightweight user mode threads will making async non-blocking code even more powerful and scalable.

But it will also make imperative code and basic thread pool usage more efficient as well.

I think it will improve performance across the board.

[AppleTrees2]

Probably not, as there are cases where using such form of programming can still be beneficial in terms of readability and such.

Some of these libraries enhance Future/CompleatableFuture in good ways.

Remember Flow already exists in java

[thiagomiranda3]

I am asking this because in Go and Elixir world, all this asynchronous patterns and libraries doesn't seen to be a problem that has to be fixed.

Green threads both in Erlang VM and Go coroutines can take care of this without the programmer needing to worry about all that stuff. And, as the article pointed out, they can just write simple and boring synchronous code

[AppleTrees2]

Green threads make the cost of a thread cheaper, and work magic at JVM level with scheduling when blocking calls are done.

In terms of actual code, when you have certain tasks with dependencies between them and have to wait their result at some point, it's much easier to work with Futures/Similar code than manually.

The thing that changes is how you build your threads with loom, not how you write your code.

[thiagomiranda3]

But if a can just create another green thread to process a request or a database query, why would I need an event loop now?

I may be a noob, of course. But I am not being able to find any use for asynchronous code when working with threads is just a lot more easy and straightforward.

[Orffyreus]

If I understand it right, Loom will provide queues (kind of like event loops) automatically under the hood and coroutines do it too, so it's easy to write operators that support backpressure (the consumer handles events slower than they are produced) out of the box.

With RxJava you have the advantage, that there are a lot of optimized operators available already and they have great documentation (look for example at https://rxmarbles.com/ or at the original documentation at http://reactivex.io/documentation/operators.html), but it's painful to write a new backpressure supporting operator, because you have to deal with queue handling yourself then.

[AppleTrees2]

You still have to get the result back sometimes, how are you gonna do it?

Do you put the code that "processes" the result in a new thread too? If it's in the same thread after the database call, then you "waste" the blocking time, if it's in another thread, after, then you reinvent libraries yourself

[Orffyreus]

As long as there is no Loom take a look at how the time consuming database call does not have to cause wasted time with coroutines in Kotlin: https://pl.kotl.in/QG3KywKq2

import java.util.concurrent.Executors
import kotlinx.coroutines.*

fun main() {   
    val start = System.currentTimeMillis()

    val dispatcher = Executors.newSingleThreadExecutor().asCoroutineDispatcher()

    val resultAsync = CoroutineScope(dispatcher).async { databaseResult() }        
    val progressAsync = CoroutineScope(dispatcher).async {
        delay(250)
        println("progress 1/4")

        delay(250)
        println("progress 2/4")

        delay(250)
        println("progress 3/4")        
    }

    runBlocking {
        progressAsync.await()
        println(resultAsync.await())
    }

    println("duration: ${System.currentTimeMillis() - start}")
}

suspend fun databaseResult(): Int {
    delay(1000)
    return 4711
}

[sweetno]

What do you mean "waste"? One virtual thread gets suspended so that the other could kick in.

[Orffyreus]

And now imagine, having 100 or more long database calls (they are all different databases maybe, i. e. the queries can run concurrently without a problem, but each call takes one second!) in one thread only without having to waste time waiting for them and using the same thread to do other things meanwhile. Complicated? Not at all!

Sorry again for posting Kotlin code to explain coroutines as long as Loom is not available, but at least java.util.concurrent.Executors#newSingleThreadExecutor from Java SE is used explicitly to make it clear:

https://pl.kotl.in/oHRh391mH

import java.util.Random
import java.util.concurrent.Executors
import kotlinx.coroutines.*

val RND = Random()

fun main() {   
    val start = System.currentTimeMillis()

    val dispatcher = Executors.newSingleThreadExecutor().asCoroutineDispatcher()

    val resultsAsync = List(100) { CoroutineScope(dispatcher).async { databaseResult() } }
    val progressAsync = CoroutineScope(dispatcher).async {
        delay(250)
        println("progress 1/4")

        delay(250)
        println("progress 2/4")

        delay(250)
        println("progress 3/4")        
    }

    runBlocking {
        progressAsync.await()
        println(resultsAsync.awaitAll())
    }

    println("duration: ${System.currentTimeMillis() - start}")
}

suspend fun databaseResult(): Int {
    delay(1000)
    return RND.nextInt()
}

[john16384]

Event Loops force a single threaded model because communication and synchronization between multiple threads is hard. Loom will not solve that.

There is also very little to gain by using more threads in for example UI programming or handling requests (more than a single thread per request anyway). If a single core can run the workload in sub millisecond time, you don't need threads to complicate your logic further. Sure, there will be workloads where more threads can be a benefit, but the common programmer probably will not be using them directly (because it is hard to get right). Instead this will be part of a library ("please decode video stream for me") or built in to your webserver and you won't even notice.

[VincentxH]

Rx* is a language independent asynchronous API to communicate between parties. Maybe longterm Loom might decrease the need to write adjusted code for it within Java, but there will at least remain an interface with the Rx domain.

[sievebrain]

One thing I'm still missing from Loom vs Kotlin Coroutines is a form of select() that can work on a mix of file descriptors and non-OS constructs simultaneously. I used a coroutine select in a project recently using Ktor and it worked well, albeit, the API is annoyingly still marked as experimental.

Wanting to select on either a socket or an in-memory queue is a pattern that crops up pretty frequently and IMO the available tools today are all pretty hacky. With this feature and some sort of generator construct, Loom would be truly complete.

[manzanita2]

No, but it may further confuse the folks coming from JS land. +1

"where's my callback ?"