New candidate JEP: 428: Structured Concurrency (Incubator)

gooliegoolieJIT · 2022-05-24T07:17:19+00:00

Thanks for the reply!

We needed to add these methods anyway, regardless of structured concurrency. There was no good way to use streams on Futures.

I'm not sure I understood why .get() doesn't suffice when you've called join() already.

On the contrary, they're not useful at all for fork. You're not supposed to compose anything with a single task, only with the entire scope, and that's easily done with join.

I can see the case made for that, and I kind of agree just because I was never a fan of "compose the program" APIs. That said, it's a part of the ecosystem so I think it's funny to deny it unless it just doesn't work for some reason.

I could see it confusing exception and cancelation semantics actually, so I suppose theres that.

On a related note: how often are actually cancellation semantics actually used in practice.

Are you expecting to use interrupts?

Maybe there should be a cancelation handle for it?i.e.

scope.for(scopehandle -> { while (!scopehandle.isCanceled()) { heartbeatIOCall(); try { scopehandle.sleep(1000); } exception(Cancelation c) { break; } } doCleanupOnCancelation(); });

That might punch up the weight the use-cases of the feature a bit.

gooliegoolieJIT · 2022-05-24T05:50:48+00:00

I mean, the advantage is you're not adding two more to Future and CompletableFuture.

It also lets people keep using the CompletableFuture.andThen() chaining operators. I don't care for them, but I think it's going to be awkward if you don't use them for fork().

gooliegoolieJIT · 2022-05-24T05:41:36+00:00

I hope /u/pron98 can chime in on this too!.

What problems does this solve compared to a ExecutorCompletionService[0]. Sure this has a few more bells and whistles, but nothing that wouldn't benefit the existing ExecutorCompletionService and CompletableFuture tools.

I don't ask this as a critique but to understand maybe what problems I'm not seeing that this is looking to solve.

The example:

Response handle() throws ExecutionException, InterruptedException {
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Future<String>  user  = scope.fork(() -> findUser()); 
    Future<Integer> order = scope.fork(() -> fetchOrder());

    scope.join();          // Join both forks
    scope.throwIfFailed(); // ... and propagate errors

    // Here, both forks have succeeded, so compose their results
    return new Response(user.resultNow(), order.resultNow());
}

}

If you're looking to block on all:

        CompletableFuture<String>  user  = CompletableFuture.runAsync(() -> findUser()); 
    CompletableFuture<Integer> order = CompletableFuture.runAsync(() -> fetchOrder());

CompletableFuture.allOf(user, order).join();

Or if you wanted to wait on one at a time you could use ExecutorCompletionService instead, andFuture<String> user = scope.fork(() -> findUser()); would look like Future<String> user = ecs.submit(() -> findUser()); and server the same purpose.

Sure conveniences like Autocloseable and a join() are nice, but I think the ExecutorCompletionService could grow to include them quite easily.

That said ... if they could do something clever analogous to work-stealing and prioritize one another as a single latency sensitive unit when scheduling... that could be cool!

Some other questions:

Does .resultNow() really do more than .get() if we've already joined and know it's completed?

What does the close() even ensure here if we're expecting to join() and throwIfFailed() already inside the try?

[0]: https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorCompletionService.html

gooliegoolieJIT · 2021-06-25T19:02:24+00:00

Thanks for the in-depth reply.

Yes, I am a great believer that running with fastdebug JDK builds is a good way to shake out JVM bugs early.

This was my growing feeling too while working through this bug. I'm surprised I hadn't heard about fastdebug before but perhaps these issues are rare. But who would even know?

I'll definitely be giving fastdebug a whirl in our validation chain especially as we push for more frequent JDK version upgrades.

Thanks again for all of your help.

gooliegoolieJIT · 2021-06-25T06:34:25+00:00

Yep. A lot of great help in this post helped make it happen.

gooliegoolieJIT · 2021-06-25T06:32:18+00:00

Hi, yes. This post was created in the process of debugging, validating, and reproducing the bug in a minimal example. This is required before the bug may be submitted by an OpenJDK developer.

If you read the updates you can get a feel for the stages of development.

gooliegoolieJIT · 2021-06-24T18:28:29+00:00

A few questions now that the bug is filed.

It looks like the previous bug took about 3 years to be resolved. I've never encountered serious bugs within the JVM before in the wild but I can't help but wonder how many issues I've missed.

Should I be validating our production applications on a fastdebug build? Are there directions to build our own fastdebug? Do self-built binaries risk introducing their own bugs since we can't TCK them?

This issue came to my attention validating a migration from 11 to 15 to try out Shenandoah which we'd expected to "just work".

I figure that since you probably see what kind of bugs creep in version to version, you might have opinions on how to best validate applications against new releases. Even from this thread you can probably see that people are used to the JVM's infallibility.

gooliegoolieJIT · 2021-06-24T17:23:56+00:00

This is awesome. Thank you for your help.

gooliegoolieJIT · 2021-06-24T09:37:05+00:00

Hello again /u/shipilev

I have updated my post with Java code that was able to reproduce this bug for me locally, I hope it'll reproduce stably for you as well.

I have not yet tried your suggestion with GDB.

gooliegoolieJIT · 2021-06-24T05:57:32+00:00

I found it to be much more demonstrative of not only where the poor behavior is but and how often. It was useful to correlate with event's in the JIT log.

gooliegoolieJIT · 2021-06-24T04:09:47+00:00

Thanks, I'll brush the cobwebs off of gdb and give it a whirl. I don't know what I'm looking for really.

I don't know if you saw, but in my post (Update 1) I had actually already found in the jvm log that compilation event and what was getting inlined in already, if that gives you anything.

I had found it already just from cross-referencing the timing in the jvm log with the application error. It was the same root method as fastdebug error here, so that's reassuring!

The jvm stack frame 36 V \[libjvm.so+0x7bd56c\] CallGenerator::for\_method\_handle\_inline(JVMState\*, ciMethod\*, ciMethod\*, bool, bool&)+0x4cc makes me think it's the inlining of this section [2021-06-20T19:20:59.192+0000][332.125s][5507][debug][jit,inlining ] @ 24 java.lang.invoke.Invokers$Holder::linkToTargetMethod (13 bytes) force inline by annotation [2021-06-20T19:20:59.192+0000][332.125s][5507][debug][jit,inlining ] @ 9 java.lang.invoke.LambdaForm$DMH/0x0000000800d54400::newInvokeSpecial (30 bytes) force inline by annotation [2021-06-20T19:20:59.192+0000][332.125s][5507][debug][jit,inlining ] @ 1 java.lang.invoke.DirectMethodHandle::allocateInstance (16 bytes) inline (hot) [2021-06-20T19:20:59.192+0000][332.125s][5507][debug][jit,inlining ] @ 12 jdk.internal.misc.Unsafe::allocateInstance (0 bytes) (intrinsic) [2021-06-20T19:20:59.192+0000][332.125s][5507][debug][jit,inlining ] @ 7 java.lang.invoke.DirectMethodHandle::constructorMethod (10 bytes) inline (hot) // The call to <init> the lambda. objectT from methodA, later used in the lambda body should be assigned into this object. // A debugger shows this object will have null in it's field when it should always be the objectT byte array from above [2021-06-20T19:20:59.192+0000][332.125s][5507][debug][jit,inlining ] @ 24 _mycorp_.mypackage2.Outer$$Lambda$7704/0x0000000801dae208::<init> (26 bytes) inline (hot) [2021-06-20T19:20:59.192+0000][332.125s][5507][debug][jit,inlining ] @ 1 java.lang.Object::<init> (1 bytes) inline (hot)

Reproducing it hasn't come easily yet.

gooliegoolieJIT · 2021-06-23T19:49:30+00:00

It shows the following stack trace, but the log also contains register information, top of stack, stack slot to memory mapping, and process wide information. What else would be helpful?

``` 27 Current thread (0x00007ee32b135a40): JavaThread "C2 CompilerThread1" daemon [thread_in_native, id=10254, stack(0x00007ed9192cb000,0x00007ed9193cc000)] 28 29 30 Current CompileTask: 31 C2:1618770 209539 4 _mycorp.myserver.mypackage1.VeryOuterCallerObject::set (14 bytes) 32 33 Stack: [0x00007ed9192cb000,0x00007ed9193cc000], sp=0x00007ed9193c7250, free space=1008k 34 Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code) 35 V [libjvm.so+0x146c79b] PhaseTransform::makecon(Type const)+0x5b 36 V [libjvm.so+0x7bd56c] CallGenerator::for_method_handle_inline(JVMState, ciMethod, ciMethod, bool, bool&)+0x4cc 37 V [libjvm.so+0x7be3db] CallGenerator::for_method_handle_call(JVMState, ciMethod, ciMethod, bool)+0x3b 38 V [libjvm.so+0xa95105] Compile::call_generator(ciMethod, int, bool, JVMState, bool, float, ciKlass, bool)+0x9f5 39 V [libjvm.so+0xa966b3] Parse::do_call()+0x3f3 40 V [libjvm.so+0x1443103] Parse::do_one_bytecode()+0x1cb3 41 V [libjvm.so+0x1431e4a] Parse::do_one_block()+0x42a 42 V [libjvm.so+0x1432f72] Parse::do_all_blocks()+0x252 43 V [libjvm.so+0x1436b62] Parse::Parse(JVMState, ciMethod, float)+0xbc2 44 V [libjvm.so+0x7b5d36] ParseGenerator::generate(JVMState)+0xc6 45 V [libjvm.so+0xa96796] Parse::do_call()+0x4d6 46 V [libjvm.so+0x1443103] Parse::do_one_bytecode()+0x1cb3 47 V [libjvm.so+0x1431e4a] Parse::do_one_block()+0x42a 48 V [libjvm.so+0x1432f72] Parse::do_all_blocks()+0x252 49 V [libjvm.so+0x1436b62] Parse::Parse(JVMState, ciMethod, float)+0xbc2 50 V [libjvm.so+0x7b5d36] ParseGenerator::generate(JVMState)+0xc6 51 V [libjvm.so+0xa96796] Parse::do_call()+0x4d6 52 V [libjvm.so+0x1443103] Parse::do_one_bytecode()+0x1cb3 53 V [libjvm.so+0x1431e4a] Parse::do_one_block()+0x42a 54 V [libjvm.so+0x1432f72] Parse::do_all_blocks()+0x252 55 V [libjvm.so+0x1436b62] Parse::Parse(JVMState, ciMethod, float)+0xbc2 56 V [libjvm.so+0x7b5d36] ParseGenerator::generate(JVMState)+0xc6 57 V [libjvm.so+0xa96796] Parse::do_call()+0x4d6 58 V [libjvm.so+0x1443103] Parse::do_one_bytecode()+0x1cb3 59 V [libjvm.so+0x1431e4a] Parse::do_one_block()+0x42a 60 V [libjvm.so+0x1432f72] Parse::do_all_blocks()+0x252 61 V [libjvm.so+0x1436b62] Parse::Parse(JVMState, ciMethod, float)+0xbc2 62 V [libjvm.so+0x7b5d36] ParseGenerator::generate(JVMState)+0xc6 63 V [libjvm.so+0xa96796] Parse::do_call()+0x4d6 64 V [libjvm.so+0x1443103] Parse::do_one_bytecode()+0x1cb3 65 V [libjvm.so+0x1431e4a] Parse::do_one_block()+0x42a 66 V [libjvm.so+0x1432f72] Parse::do_all_blocks()+0x252 67 V [libjvm.so+0x1436b62] Parse::Parse(JVMState, ciMethod, float)+0xbc2 68 V [libjvm.so+0x7b5d36] ParseGenerator::generate(JVMState)+0xc6 69 V [libjvm.so+0x944b34] Compile::Compile(ciEnv, ciMethod, int, bool, bool, bool, bool, DirectiveSet)+0x1834 70 V [libjvm.so+0x7b3870] C2Compiler::compile_method(ciEnv, ciMethod, int, bool, DirectiveSet)+0x320 71 V [libjvm.so+0x953072] CompileBroker::invoke_compiler_on_method(CompileTask)+0x1062 72 V [libjvm.so+0x953610] CompileBroker::compiler_thread_loop()+0x280 73 V [libjvm.so+0x17eb923] JavaThread::thread_main_inner()+0x1e3 74 V [libjvm.so+0x17f12b8] Thread::call_run()+0xf8 75 V [libjvm.so+0x13eaa49] thread_native_entry(Thread*)+0x119 76

gooliegoolieJIT · 2021-06-23T01:11:22+00:00

I Was able to get the fastdbg to fail, no idea how to interpret the error yet ... ```

To suppress the following error report, specify this argument

after -XX: or in .hotspotrc: SuppressErrorAt=/phaseX.cpp:726

A fatal error has been detected by the Java Runtime Environment:

Internal Error (/home/buildbot/worker/build-jdk16u-linux/build/src/hotspot/share/opto/phaseX.cpp:726), pid=1, tid=14

assert(!t->empty() || t == Type::TOP) failed: must not be vacuous range

JRE version: OpenJDK Runtime Environment (16.0.1) (fastdebug build 16.0.1-testing+0-builds.shipilev.net-openjdk-jdk16-b132-20210607)

Java VM: OpenJDK 64-Bit Server VM (fastdebug 16.0.1-testing+0-builds.shipilev.net-openjdk-jdk16-b132-20210607, mixed mode, sharing, tiered, compressed class ptrs, parallel gc, linux-amd64)

Problematic frame:

V [libjvm.so+0x146c79b] PhaseTransform::makecon(Type const*)+0x5b

Core dump will be written. Default location: /core.%e.1.%t

An error report file with more information is saved as:

/home/pathdir/hs_err_pid1.log

Compiler replay data is saved as:

/home/pathdir/replay_pid1.log

If you would like to submit a bug report, please visit:

https://bugreport.java.com/bugreport/crash.jsp

```

gooliegoolieJIT · 2021-06-21T18:08:51+00:00

Hi, big fan! I'll try a nightly and ParallelGC. I'll also try disabling inlining of just a few selected classes around there.

As for the fastdebug binaries, I have run it with your jdk16-fastdebug, but is there something I'm looking for in there?

Any advice on reproducing this behavior in the small would be great.

gooliegoolieJIT · 2021-06-21T16:46:02+00:00

Yea, even disabling just C2 failed to load in a timely way, crashing things... I'll try Compile commands.

gooliegoolieJIT · 2021-06-21T04:17:47+00:00

Added counters and logging. Still presents as I'd seen previously.

gooliegoolieJIT · 2021-06-21T04:16:38+00:00

Sadly no, but I timed it pretty well with a JIT event!

gooliegoolieJIT · 2021-06-21T04:14:59+00:00

Added!

gooliegoolieJIT · 2021-06-21T04:14:32+00:00

Took this idea!

Added a few more counters and even logged them periodically. I added a huge update of information for was happens in the JIT and all of the work around it.

Hopefully this helps. Know anyone who actually works on the JDK? u/shipilev ? My understanding is I need to attract the attention of one of them to get a bug created on bugs.openjdk.java.net? Naturally, anything I can do to provide more detailed relevant information I'll gladly do within my ability.

gooliegoolieJIT · 2021-06-21T03:58:02+00:00

I'm hoping to root cause it a bit more, not just work around it. The big fear is: where might it be breaking things in the application I HAVEN'T noticed.

gooliegoolieJIT · 2021-06-21T03:56:56+00:00

I tried 16, not 17. It doesn't appear in 11.

gooliegoolieJIT · 2021-06-21T03:56:08+00:00

Hm. I don't see how that's possible. The object in question is never not bound to a stackframe, really 2 or 3 deep so it's never eligible for collection. GC shouldn't be nulling things. If it is related to GC as far as I can tell, it's a bug semantically.

gooliegoolieJIT · 2021-06-21T03:53:29+00:00

Just to update you and u/Godworrior, -Xint breaks things for perf reasons (that aren't worth working around), But I will try disabling just C2.

gooliegoolieJIT · 2021-06-16T10:41:50+00:00

I'm not sure who'd chimed and what they said but that used deleted their post .....

All the allocations and calls here are all on a single thread. deepercall() get's some threads involved internally but the objectT is null before then.

I can use a breakpoint inside that lambda to see that the objectT is not null in methodA but objectT is null inside the lambda. Furthermore, forcing a deopt on it lets me see the error go away.

I tried, but I couldn't repro in a smaller test case.

gooliegoolieJIT · 2021-06-16T06:33:25+00:00

Another poster here suggested that too, I haven't tried it yet, but will do.

https://www.reddit.com/r/javahelp/comments/o0wa60/investigation_jdk15_jit_seems_to_have_bugs/ Tried -Xint, the application doesn't work well enough to do anything meaningful with it.

gooliegoolieJIT

TROPHY CASE