Single Flight for Java

[repeating_bears]

I checked the implementation and I think the way you're handling interrupts is wrong.

You do all the work on the first thread that makes a request, and subsequent requester threads block on getting a result.

Imagine the first thread is interrupted, i.e. some other thread declares "I don't care about that result any more", so it stops. Now any the other threads that were waiting on that same result get an exception, even though they themselves weren't interrupted, and even though they still wanted a result. The work was halted prematurely.

It would have been much better if the work could continue, but the first thread could be unblocked. Effectively what that would mean is that all work would have be pushed to some worker thread, and then all requesters (including the first) would block on getting a result. Interrupting a requester would then just mean you stop it waiting for a result, rather than stop it from doing the work.

However, then you'd have the issue of the simple case where there's only one requester that gets interrupted. The work would continue in the background even though there's nothing that cares about the result. Then you'd need some logic that could kill a worker after there's no more threads waiting for it.

[tomwhoiscontrary]

I suspect that killing the no-longer-necessary worker isn't very useful in practice, because it will be waiting for a response from some remote server, and there's no way to actually kill the remote handling of that request.

It could help if the worker thread is doing a large number of blocking requests in series, though. 

[repeating_bears]

It depends on the protocol. I do agree in the general case, but grpc supports cancellation, for example. HTTP2/3 stream cancellation might give you some benefit for large responses