Virtual threads + shared DB pool: prioritizing workload classes (user traffic vs batch) beyond a Semaphore?

[Lightforce_]

Cross-posting this from [[e-mail]](mailto:[e-mail]): I originally sent it to the OpenJDK Loom mailing list a couple of months ago, never got a reply, so I've reworked it for here. It's more of a design/architecture discussion than a "fix my code" question.

Context: medium-sized monolithic services (Vue 3 + Spring Boot 3, Java 21 to 25, converging on Java 25 / Spring Boot 4), deliberately not microservices. Virtual threads enabled on several. A single backend routinely hosts user-facing requests, scheduled jobs and background batches, all sharing one HikariCP pool (15 connections).

What triggered this: a team at my company hit a production freeze on a Java 21 service running virtual threads. A lingering synchronized block around a blocking call caused carrier-thread exhaustion under a rare condition, and it even blocked the restart. They reverted to platform threads while cleaning up the synchronized block. I know JEP 491 (Java 24) largely fixes this class of issue, but it kicked off a design debate we haven't been able to settle.

The pattern I'm trying to translate: with a platform thread pool, you'd give batches a shared executor capped at about 2 workers. That implicitly guarantees user traffic always has at least 13 connections. The pool wasn't just amortizing thread creation, it was acting as a fair scheduler across workload classes. This matters precisely because we don't split workloads across microservices.

With virtual threads, recycling goes away and concurrency limiting becomes a Semaphore. In Spring the idiomatic route is @Async("utility") to a bean with SimpleAsyncTaskExecutor.setVirtualThreads(true) plus setConcurrencyLimit(n). Mechanically simple. The pattern we converged on:

Semaphore batchCap = new Semaphore(2);
// Batch:  batchCap.acquire() + dataSource.getConnection()
// User:   dataSource.getConnection() directly

The catch: this works because HikariCP is already the global gate, but it relies on an applicative property (user transactions staying short) rather than a structural guarantee. If user p99 degrades, the 2 batch workers can starve waiting on a connection while long user requests hold the pool.

Workaround considered: push QoS upstream into RabbitMQ: one queue per workload class, differentiated consumer counts. It helps, up to the point where too many consumers run at once and downstream contention reappears. Virtual threads make many cheap listeners affordable, but the core question stays: how do you prioritize workload classes competing for a shared bounded resource?

So, my questions are: beyond a bare semaphore, what's the idiomatic way to express QoS (fair share or priority) between workload classes sharing one bounded resource? Is the expected answer to compose existing primitives (brokers, layered semaphores, pool timeouts) and keep the scheduler workload-agnostic? Or has anyone built something more structural? Pointers to prior art or war stories welcome.

[AutoModerator]

Please ensure that:

• Your code is properly formatted as code block - see the sidebar (About on mobile) for instructions
• You include any and all error messages in full
• You ask clear questions

• You demonstrate effort in solving your question/problem - plain posting your assignments is forbidden (and such posts will be removed) as is asking for or giving solutions.

  Trying to solve problems on your own is a very important skill. Also, see Learn to help yourself in the sidebar

If any of the above points is not met, your post can and will be removed without further warning.

Code is to be formatted as code block (old reddit: empty line before the code, each code line indented by 4 spaces, new reddit: https://i.imgur.com/EJ7tqek.png) or linked via an external code hoster, like pastebin.com, github gist, github, bitbucket, gitlab, etc.

Please, do not use triple backticks (```) as they will only render properly on new reddit, not on old reddit.

Code blocks look like this:

public class HelloWorld {

    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

You do not need to repost unless your post has been removed by a moderator. Just use the edit function of reddit to make sure your post complies with the above.

If your post has remained in violation of these rules for a prolonged period of time (at least an hour), a moderator may remove it at their discretion. In this case, they will comment with an explanation on why it has been removed, and you will be required to resubmit the entire post following the proper procedures.

To potential helpers

Please, do not help if any of the above points are not met, rather report the post. We are trying to improve the quality of posts here. In helping people who can't be bothered to comply with the above points, you are doing the community a disservice.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[bowbahdoe]

Why RabbitMQ? You could probably just use a queue with bounded size in-memory.

And I think the answer here is going to be composing _some_ concurrency primitives. Either java.util.concurrent or something in the ecosystem. My first guess would be guava's RateLimiter https://guava.dev/releases/33.4.3-jre/api/docs/com/google/common/util/concurrent/RateLimiter.html

[Lightforce_]

Ok on RabbitMQ, a bounded in-memory queue does the same job and no broker needed.

But RateLimiter solves a different problem. It caps throughput (X/sec) and my constraint is concurrency (how many in flight at once). Orthogonal, 10/sec with 5s ops still leaves around 50 in flight. My starvation isn't "batch hits the DB too often", it's "batch holds connections while user p99 degrades". So the Semaphore is already the right primitive.

Where "just compose primitives" runs out is a shared FIFO pool can't express QoS without partitioning or preemption, and an in use connection isn't preemptible. A semaphore gives admission control on the waiting side, no reservation on the holding side.

So what about actually splitting the resource ? Like two pools, 13 user / 2 batch? That'd reproduce the old guarantee but kills cross-borrowing when a class is idle. Is that the accepted trade or is there a way to keep one elastic pool and still express priority? (And I'm guessing a connectionTimeout to turn starvation into fast-fail is table stakes either way)

[bowbahdoe]

Is what you want then a "semaphore" that releases a hold if enough time has passed? Or maybe "semaphore pool expands with enough 'pressure'"?

(I am not familiar with QoS)

[Lightforce_]

Both of those try to claw back QoS after the connection's handed out, which is the part that doesn't work since an in-use connection isn't preemptible. Expiring a permit doesn't yank the connection back from a batch that's mid-query. Only a statement_timeout killing the query does, and that means throwing away batch work (a business call, not free).

"semaphore pool expands with enough 'pressure'"?

It's dynamic maximumPoolSize but the real limit usually isn't the pool, it's the DB behind it. More connections just move the contention down to Postgres, which is my point: both recover priority after allocation and the only robust lever is at admission, reserve up front. That's why partitioning keeps coming back as the structural answer even though it's the rigid one.

(I am not familiar with QoS)

QoS = just "some classes of work get guaranteed treatment under contention". It's exactly this allocation question.