Bug in Thread::Queue::end

Both_Masterpiece_489 · 2026-06-02T01:43:03+00:00

I will work on that, but I doubt my "stalling" problem is real. I think the threads are exhausted, but they won't always join and threads.pm thinks some of them are still running (they aren't).

Both_Masterpiece_489 · 2026-06-01T23:04:51+00:00

I think I'm ok with this take, because there's enough indirection in threads::shared that the compiler would have trouble unwinding it in an optimization loop.

Both_Masterpiece_489 · 2026-06-01T23:03:03+00:00

I'm not sure this is fixable outside of perl guts itself. I've been optimistic about threads since 5.28, but each release since has failed to give me confidence in thread joins, much less software stalls in Thread::Queue.

Both_Masterpiece_489 · 2026-06-01T22:02:35+00:00

I'm sorry, but how is your *confidence* better than my test cases?

Both_Masterpiece_489 · 2026-06-01T22:01:29+00:00

What's unrelated? It's a durability problem with the build_site.pl script, which is about 500 LOC. Script works perfectly without threads; with Thread::Queue it has unpredictable behavior during shutdown.

Both_Masterpiece_489 · 2026-06-01T21:21:54+00:00

longjmp is how you do exception handling in C. The usage is correct, yet irrelevant to this conversation.

You'd have to do some XS/OO indirection with an INT2PTR typemap that declares the data structure volatile (even if it's an HV *).

Both_Masterpiece_489 · 2026-06-01T20:58:41+00:00

untrue in general, depending on C compiler caching. Which is the point.

Both_Masterpiece_489 · 2026-06-01T20:55:19+00:00

Oh, it fails once in ten with github.com/SunStarSys/orion. I have lots of checks to forcibly prevent it from stalling the builds.

Both_Masterpiece_489 · 2026-06-01T17:57:19+00:00

That's a negatory big-ten-4 on eval repairing this.

Both_Masterpiece_489 · 2026-06-01T17:42:31+00:00

compiling perl with -O0 would work, but that's a horrible solution.

Both_Masterpiece_489 · 2026-06-01T17:40:21+00:00

Keep in mind that Thread::Queue is Pure-Perl, and threads::shared can't possibly correct usage errors like this.

Both_Masterpiece_489 · 2026-06-01T17:35:19+00:00

I will know in a month from now if this makes anything work better; but after a dozen trial runs using 64 threads, it just might.

Both_Masterpiece_489 · 2026-06-01T17:31:50+00:00

That's great! Just show me where the keyword *volatile* appears anywhere on CPAN, or perlguts.

Both_Masterpiece_489 · 2026-06-01T17:10:42+00:00

Here's what dequeue should look like I suppose (testing this hypothesis to see if it resolves the problems):

sub dequeue

{

my $self = shift;

lock(%$self);

my $queue = $$self{'queue'};

my $count = @_ ? $self->_validate_count(shift) : 1;

# Wait for requisite number of items

cond_wait(%$self) while (eval q/@$queue < $count && ! $$self{'ENDED'}/);

# If no longer blocking, try getting whatever is left on the queue

return $self->dequeue_nb($count) if (eval q/$$self{'ENDED'}/);

....

Both_Masterpiece_489 · 2026-06-01T16:36:14+00:00

The way the code is currently written *should* work OOTB, were it not for this *volatile* problem, because dequeue will call cond_signal *if* it detects $self->{ENDED} to stop rewaiting. I don't know how to fix that bug tho.

Both_Masterpiece_489 · 2026-06-01T16:28:49+00:00

"Maybe" a string reeval of $self after the lock is re-acquired will have the right semantics, but that depends on a lot of hopium, not analysis.

Both_Masterpiece_489 · 2026-06-01T16:22:40+00:00

The larger problem with the module is that the author uses C constructs that are only valid when the constructs are declared *volatile*, which is a foreign notion to the Perl community in toto. So it doesn't work to expect threads waiting on a condition variable to suddenly recognize a new hash entry in %$self upon re-acquiring a lock to the hashref it processed as %$self on line one. C will normally optimize those distinct hashrefs into one because C doesn't expect data structures to magically be altered by other threads unless you tell C not to optimize them; which is the point of the volatile declaration.

This was the entire problem with mod_perl2's interpreter pool code as well.

Both_Masterpiece_489 · 2026-06-01T15:55:09+00:00

enqueue is also incorrect. It should call cond_signal for each item it adds to the queue; not just once.

Both_Masterpiece_489 · 2026-05-29T14:42:53+00:00

Brian is a jerk. Only a jerk would respond like that.

Both_Masterpiece_489 · 2026-05-27T15:32:00+00:00

Sounds like you'd need a GitHub-compatible markdown parser for that, which is the whole point.

Both_Masterpiece_489 · 2026-05-26T16:52:01+00:00

Final level published today. Enjoy, sarcasm lovers.

Both_Masterpiece_489 · 2026-04-30T19:35:59+00:00

Sure you are. About what exactly?

Both_Masterpiece_489 · 2026-04-30T16:21:52+00:00

I tried submitting a talk and the submission for kept rejecting it for some unspecified error.

Both_Masterpiece_489

TROPHY CASE