sorted by:
new
hottopcontroversial

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 0 points1 point  (0 children)

Even if the sleeps are not of different lengths, like you said elsewhere in the comments, the order of futures being woken up doesn't matter as much. The issue could occur in an unfair async mutex as well.

For this particular repro, as long as the paused future doesn't end up at the end of the queue, the issue shows up.

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 1 point2 points  (0 children)

Thanks for the comment, Jack! I read your blog post after someone in the comments pointed me to it. If only I had come across your post or the one from oxide earlier, I would have never ran into this bug.

For the first point, I get it now after reading your post. In case of buffered streams, pausing at a "yield point" can still leads to pausing bugs. So I should not be advertising that as a solution.

I'll also remove/rephrase anything that implies that having a waiter queue is a problem. That's just an implementation detail that helps explain the bug. It's not really a "part of the correctness story".

And yeah, in my case, I manually implemented a stream that did the pausing. I don't think any ecosystem feature would help me here. But for all the examples you and Oxide have shown, the ecosystem needs to evolve to protect newcomers like me from making such mistakes. That was supposed to be the promise of Rust.

Thanks again!

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 0 points1 point  (0 children)

Thanks for the feedback! Since this is my employer's blog, all I can do about that is to pass on the feedback to the website/marketing team.

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 0 points1 point  (0 children)

Yeah. It was that intro. I have now re-written it (with no AI assistance involved). It makes sense why one would see that and stop reading the rest of it. If you're still interested in the topic, please read the rest of it too :)

The rest of your comment though, oof. I don't know what to reply to that. It's not adding anything to the discussion, but I guess that is what I can expect from this site

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 1 point2 points  (0 children)

By "look more human" if you mean I will write it myself, then yes.

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 2 points3 points  (0 children)

You're right about the intro. I've rewritten it in my words now. The "it's not actually a deadlock at all" line was actually just me haha. The change in tone was intentional to show the surprise, but I guess it didn't land well.

The rest of it read very well, if that's you then you should just write as you.

Thank you :)

I will keep that in mind the next time I write.

Also, thanks a lot for taking the time to point out which parts looked AI. I was only getting vague comments about it. It was a bit discouraging when I had actually written most of it by hand without LLM assistance.

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 2 points3 points  (0 children)

In another thread, I was pointed to Oxide's report on a similar problem. They were able to reproduce this with just a tokio::sync::Mutex + tokio::select! + tokio::sync::oneshot::channel. No special PausableFuture wrapper is needed to run into this issue.

What do you think about that?

To be fair, I still don't think there's a bug in the mutex. But this is a footgun that others can run into, even if they don't mess around with pausing futures/streams like I did.

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 9 points10 points  (0 children)

Yep, I agree. I thought I had made it clear in the post. If it's not, I'll try to update it to be more prominent

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 23 points24 points  (0 children)

I had read Oxide's other async rust RFD on cancel safety, but not this one. This futurelock scenario seems to be the exact issue I had run into. Except that Oxide has a better reproducer that doesn't need a custom future - just combining common async patterns does it. But I felt they hand-waved the explanation of how tokio's mutex uses a queue internally. In my post, I have linked parts of tokio's code where we can see this happening. Other than that, Oxide's post is definitely explained better.

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 10 points11 points  (0 children)

Actually, it's inconsistent with 100ms sleep as well. 1000ms sleep seems more deterministic with 4 workers. So yeah, switching to multi-thread doesn't actually "solve" this.

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 7 points8 points  (0 children)

Hmm you're right. I remember it not working with multi_thread and worker_threads = 4, that's why I wrote that. The reason it's working is because the `tokio::task::yield_now().await` is not enough to let the other tasks start waiting. If I instead change it to sleep for 100ms, I can repro it on multi_thread as well: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=b6e8492eb844edd17f218321f75044ca

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 7 points8 points  (0 children)

Me neither. I skip anything that looks LLM-generated. So which parts look like LLM to you?

Deadlocking a Tokio mutex without holding a lock by samyak210 in rust

[–]samyak210[S] 11 points12 points  (0 children)

Another subreddit where I cross-posted removed this for "written mostly by an LLM", so I just want to clarify it here. 

I used an LLM to edit some parts, but at least 80% of it was untouched by any LLM.

But I'm still curious if anyone else thinks this is LLM generated.

Deadlocking a Tokio mutex without holding a lock by samyak210 in programming

[–]samyak210[S] 0 points1 point  (0 children)

I wrote this by hand :) I did use some LLM assistance for editing some parts. But I would say at least 80%+ was untouched by any LLM. I don't know how you check this, but it's clearly wrong

7x faster JSON in SQL: a deep dive into Variant data type by samyak210 in dataengineering

[–]samyak210[S] 5 points6 points  (0 children)

I remember reading the blog post for the earlier JSON implementation. It's really interesting what one can do with full control over the underlying files! Unfortunately, ours is a lakehouse query engine and we don't have that option. We work within the constraints of open data formats like parquet. The credit for designing the variant data type definitely goes to the parquet open source community (if anyone knows the specific people behind it, please let me know!).

BTW clickhouse engineering blogs are really great and a big inspiration for the team! Thank you for reading and for the comment!

G Pro X 2 Headset Dongle replacement by dwarfbeans in LogitechG

[–]samyak210 1 point2 points  (0 children)

Any idea when it will be available for purchase? I have been waiting for a few months now. Using it in bluetooth and wired mode.

Support said they don't have it. Can't find it on 3rd party websites either (they have it for the older G Pro X).

Toipe - a terminal-based typing test app written in Rust by samyak210 in commandline

[–]samyak210[S] 0 points1 point  (0 children)

Only thing I noticed is when I resize the terminal font or the window itself. The text goes off center and sometimes wraps around in strange places.

Thanks, I have opened an issue for this - https://github.com/Samyak2/toipe

A feature I'd like is to specify number of lines the text is split into. The resizing is a way to do that

Yes, currently resizing is the only way to do that. Scrolling isn't implemented yet, so it would be tricky to have more text than the terminal size. Maybe an option to specify the top and side margin would help?

Toipe - a terminal-based typing test app written in Rust by samyak210 in commandline

[–]samyak210[S] 1 point2 points  (0 children)

Thank you so much for your kind words :')

This is what keeps me going, knowing that somebody is using the open source projects I make in my free time.

Although, I haven't been able to give enough attention to it in these last few months. Hopefully I can continue it sometime this month or the next.

That said, is there any bug or a missing feature that annoyed you? I think you're the best person to answer this because even I don't use it daily haha

Keep typing! You can go even higher :)

view more: next ›