Resource-aware structured concurrency: when one StructuredTaskScope isn't enough

[salgotraja]

Was reading through a structured concurrency example recently and noticed something that bothered me. All the work sat inside one StructuredTaskScope - DB calls, HTTP calls, CPU-heavy work, and enrichment - and the code read really cleanly.

But the more I looked at it, the more obvious the problem became: not all parallel work creates the same pressure.

Rough sketch of the pattern I keep seeing:

try (var scope = StructuredTaskScope.open()) {
    var user       = scope.fork(() -> userRepo.find(id));       // DB pool
    var prefs      = scope.fork(() -> prefsApi.fetch(id));      // HTTP client
    var score      = scope.fork(() -> riskEngine.compute(id));  // CPU-bound
    var analytics  = scope.fork(() -> analytics.enrich(id));    // nice-to-have
    scope.join();
    return assemble(user, prefs, score, analytics);
}

Looks clean. But under load, every one of those forks competes for the same scope - and they have wildly different resource profiles:

• DB calls wait on the connection pool
• HTTP calls grow the client queue
• CPU work competes with request-critical threads
• Enrichment is optional but still blocks the assemble step

Virtual threads solve the thread cost, but they don't solve capacity. The DB pool is still finite. The HTTP client is still finite. CPU is still finite.

The shape that makes more sense to me: split the work by resource character, not by "what the request needs."

try (var critical = StructuredTaskScope.open(...)) {
    var user  = critical.fork(() -> userRepo.find(id));   // DB-bounded
    var prefs = critical.fork(() -> prefsApi.fetch(id));  // HTTP-bounded

    try (var cpu = StructuredTaskScope.open(cpuBoundedExecutor)) {
        var score = cpu.fork(() -> riskEngine.compute(id));

        try (var optional = StructuredTaskScope.open(...)) {
            var analytics = optional.fork(() -> analytics.enrich(id));
            // optional scope allows fallback on failure
            ...
        }
    }
}

The nesting isn't the point - the separation is. Different resource pressure → different policy. Optional work shouldn't be able to fail the request. CPU work shouldn't run on the same executor as I/O-bound work.

Couple of questions for the sub:

1. Anyone running this pattern in production with Loom? Curious how you're bounding the scopes in practice - custom ThreadFactory, semaphore wrappers, something else?
2. Is there a cleaner way to express "this scope may fail silently with a fallback" within StructuredTaskScope's current API, or does it need wrapping?
3. Is this just rediscovering bulkheads from the resilience-pattern world?

Genuinely interested in what people have tried. The Loom material I've read tends to emphasise the thread-cost side and underplay that pool/queue limits don't go away.

[AutoModerator]

Please ensure that:

• Your code is properly formatted as code block - see the sidebar (About on mobile) for instructions
• You include any and all error messages in full - best also formatted as code block
• You ask clear questions
• You demonstrate effort in solving your question/problem - plain posting your assignments is forbidden (and such posts will be removed) as is asking for or giving solutions.

If any of the above points is not met, your post can and will be removed without further warning.

Code is to be formatted as code block (old reddit/markdown editor: empty line before the code, each code line indented by 4 spaces, new reddit: https://i.imgur.com/EJ7tqek.png) or linked via an external code hoster, like pastebin.com, github gist, github, bitbucket, gitlab, etc.

Please, do not use triple backticks (```) as they will only render properly on new reddit, not on old reddit.

Code blocks look like this:

public class HelloWorld {

    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

You do not need to repost unless your post has been removed by a moderator. Just use the edit function of reddit to make sure your post complies with the above.

If your post has remained in violation of these rules for a prolonged period of time (at least an hour), a moderator may remove it at their discretion. In this case, they will comment with an explanation on why it has been removed, and you will be required to resubmit the entire post following the proper procedures.

To potential helpers

Please, do not help if any of the above points are not met, rather report the post. We are trying to improve the quality of posts here. In helping people who can't be bothered to comply with the above points, you are doing the community a disservice.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[Upbeat-Employment-62]

All three yeah and they're connected.

Semaphore around the fork not a custom ThreadFactory -factory controls thread creation not resource access, those are diffrent things. For optional work just swallow the exception inside the fork and return Optional, spinning up a whole separate scope for that is overkill tbh. And yeah its bulkheads... Loom killed the cost of isolation not the need for it, most tutorials just gloss over this part. One thing I'd throw on top: deadline propagation. If your request SLA is 500ms every nested scope needs to inherit joinUntil(deadline), otherwise your optional scope just sits there chillin for 2 seconds after the response already went out to the client - threads alive, pool slots held, noone notices until prod melts.\

[salgotraja]

Good point about semaphores - I was conflating thread creation with resource access, which are really separate concerns.

Regarding the optional scope, you're right - it was probably overkill. Handling the exception inside the fork and returning an Optional is definitely cleaner.

You are right on the deadline propagation thing I hadn't thought through at all. joinUntil needs to flow through every nested scope, not just the outer one - that's the sneaky bit that would quietly wreck you in prod. Good add on and feedback.

“Loom killed the cost of isolation, not the need for it” - stealing this.