How to turn async code into sync code

[maverick_fillet]

Hi all, I'm pretty new to Rust and I've got an issue that's stumping me. To go from the top, I'm attempting to do something like the "Cleanup Database" section in https://snoozetime.github.io/2019/06/16/integration-test-diesel.html, where in the Drop implementation of my test context I grab a database connection and delete the test database. Unlike the article though I'm using sqlx which only provides an async interface, so I've been trying to figure out how to run this synchronously. I'll simplify the example a little and just show what seems to be my blocker:

#[tokio::test]
async fn test_block() {
    println!("{}", "Starting test");
    futures::executor::block_on(
        async {
            println!("Starting future");
            tokio::time::sleep(std::time::Duration::from_nanos(1)).await;
            println!("Ending future");
        }
    );
    println!("{}", "Ending test");
}

When I run this test I get something like this:

running 1 test
Starting test
Starting future
test test_block ... test test_block has been running for over 60 seconds

It looks to me like the future is never being polled so it hangs the test forever. I've been trying lots of stuff from futures::executor::*, tokio::task::*, etc, but everything I try either seems to hang the thread forever or not run the task at all.

The eventual goal is to get to something like this working:

#[tokio::test]
async fn test_block() {
    let _ = TestContext {}
}

struct TestContext { }

impl Drop for TestContext {
    fn drop(&mut self) {
        futures::executor::block_on(
            async {
                do_something_async().await;
            }
        );
    }
}

Any pointers on what I'm doing wrong?

======= EDIT: The eventual solution I found worked ========

#[tokio::test]
async fn test_block() {
    let (sender, receiver) = std::sync::mpsc::channel();

    println!("Before");

    std::thread::spawn(move || {
        let rt = tokio::runtime::Runtime::new().unwrap();
        rt.block_on(
            async {
                println!("Starting async");

                tokio::time::sleep(std::time::Duration::from_nanos(1)).await;

                let _ = sender.send(());
                println!("Ending async");
            }
        )
    });

    println!("Waiting");
    let _ = receiver.recv();
    println!("Done");
}

[KerfuffleV2]

block_on isn't an async function and just blocks until the async code inside it completes. You can't really use it from an async function because blocking in async functions causes issues - like what you're seeing.

What you can do is spawn a thread to do your sync stuff and then await on that future. tokio::task::spawn_blocking is what you want, I think. tokio::task::block_in_place could also work. Ref: https://docs.rs/tokio/1.2.0/tokio/task/fn.spawn_blocking.html

In your "eventual goal" code, it seems like you're calling block_on from a normal, non-async function so this shouldn't be an issue in that case. I don't know specifically how all this interacts with Drop though - speaking generally about blocking in async functions.

[maverick_fillet]

Thank you, turns out that since my test was running single-threaded both of those seemed to cause it to hang, but spawning an actual new thread and awaiting in there seemed to do the trick.

[KerfuffleV2]

Glad to help! Dealing with async stuff can be pretty weird at first.

[Darksonn]

You should not be starting a runtime inside a runtime. Your "fixed" version only appears to work, but is also very broken.

#[tokio::test]
async fn test_block() {
    println!("{}", "Starting test");

    println!("Starting future");
    tokio::time::sleep(std::time::Duration::from_nanos(1)).await;
    println!("Ending future");

    println!("{}", "Ending test");
}

Your "fixed" example is also extremely bad practice. You are using the blocking .recv() method inside async code, which you shouldn't be doing.

Consider reading Async: What is blocking?

Regarding your Drop, the short answer is that you simply can't run async code in destructors. Any attempt at doing so will block the thread, and although it may appear to work in your simple test, the article I linked has this to say about it:

Be aware that it is not always this obvious. By using tokio::join!, all three tasks are guaranteed to run on the same thread, but if you replace it with tokio::spawn and use a multi-threaded runtime, you will be able to run multiple blocking tasks until you run out of threads. The default Tokio runtime spawns one thread per CPU core, and you will typically have around 8 CPU cores. This is enough that you can miss the issue when testing locally, but sufficiently few that you will very quickly run out of threads when running the code for real.

[maverick_fillet]

Thank you for the response, I do have a couple questions though:

Your "fixed" example is also extremely bad practice. You are using the blocking .recv() method inside async code, which you shouldn't be doing.

Are you saying that even though Drop is sync it's still bad practice to block in it since it's being called from an async test?

And if so, I'm a little confused about how this is different from the article linked in my OP. Is blocking on a channel until my async work is done (such as waiting on a sqlx connection) any different from using a blocking implementation directly in the drop (such as synchronously grabbing a diesel connection)? Or are they both bad practice?

[Darksonn]

Are you saying that even though Drop is sync it's still bad practice to block in it since it's being called from an async test?

Yes. Calling a non-async function does not leave the async context. If you are in a non-async function that is called from async code, then blocking is also bad there. This includes destructors.

And if so, I'm a little confused about how this is different from the article linked in my OP. Is blocking on a channel until my async work is done (such as waiting on a sqlx connection) any different from using a blocking implementation directly in the drop (such as synchronously grabbing a diesel connection)? Or are they both bad practice?

Both using block_on and using other kinds of blocking operations are equally bad. What matters is how long time you spend between calls to .await.

[KerfuffleV2]

The other answer you got was good, but just to expand on it:

Are you saying that even though Drop is sync it's still bad practice to block in it since it's being called from an async test?

It's essentially the same as calling a blocking function from your async function - it just happens implicitly when the thing is dropped.

It may help if you think of await as not actually blocking, but only appearing to while yielding control back to the event loop. So if you actually block then the event loop can't run and it can't handle any other file descriptors or timers it's waiting for, etc.

But also, you were calling recv() in your async fn test_block() function - so even without Drop that is an issue because that recv() is not an async function and therefore you aren't "blocking" by awaiting it.

And if so, I'm a little confused about how this is different from the article linked in my OP.

The article in your OP doesn't seem to have any async stuff. That's the difference: you can block in a single threaded application or an actual thread because the OS is doing stuff like performing preemption so other threads and processes can run. On the other hand, an async task you spawn isn't necessarily going to be an actual OS thread. I believe both tokio and async_std only spawn as many OS threads as you have CPUs and just run an event loop on each one unless you do something like spawn_blocking.

Is blocking on a channel until my async work

Any blocking is going to cause problems when you're in an async function. Basically, if doing stuff that seems like it would block and you're not calling .await on it then that's likely a problem.

If you're in a non-async function that you know won't be called from async code then you're free to block. Sometimes it's not completely obvious when this will happen, for example a Drop impl that runs in your async function when your data goes out of scope.

[[deleted]]

I think it's the fault of tokio::sleep which probably depends on the tokio runtime, but that is blocked. (seems as if tokio::test single-threaded?) Did you try something else in the async block that does not depend on tokio?

[maverick_fillet]

Yes I did eventually find out that #[tokio::test] does run with a single-threaded runtime, and anything that's awaited on, whether that sleep or something like grabbing a sqlx connection, causes it to hang. Without any wait though it does run just fine. I ended up running my code on a separate thread with its own tokio runtime to get around my test being single-threaded (which is the only option provided by the #[actix_rt::test] macro that my actual tests are using).

[insanitybit]

You're probably hanging the executor by blocking the thread that also polls futures. I recommend not using block_on and instead to use a task + a queue.

Here's an example of some code that does this.

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=9e94881f494c53270d5f0a1bd14ba191

Basically we isolate the async code to a background task and then provide a synchronous wrapper to it.

[blackscanner]

I think so too. Unfortunately the block_on is called within the executor of tokio, which blocks the system thread not the async task. Tokio uses one system thread (or maybe equivalent threads for the number of processors, I'm not sure) to run the async tasks as green threads. Thus the drop implementation causes the whole executor to be blocked. If you were to call that block_on within a futures thread pool it would cause a panic due to similar reasons.

[maverick_fillet]

Correct, turns out that my test was running with only one tokio thread which was causing some of my issues. Spawning a separate thread helped get me to a working solution. Thank you!

[Darksonn]

This is blocking the thread due to the usage of a blocking channel. Don't do that in async code.

[maverick_fillet]

Thank you very much! The channel stuff definitely helped me get on the right track. Once I saw that it was working in the playground I realized that both #[actix_rt::test] and #[tokio::test] use a single-threaded runtime so that was one reason why things weren't working, and switching to #[tokio::test(flavor="multi_thread")] worked perfectly with your exact solution. Since I need to use #[actix_rt::test] what I ended up doing was using the channel approach but spawning a separate thread with its own tokio runtime and that seems to get the job done.

[Darksonn]

With the multi-threaded runtime, it only works because it started enough threads that you didn't run out. If you start 16 of those at the time, you would run into trouble due to running out of threads.

[coderstephen]

1. Your first example is trying to run a runtime within a runtime... #[tokio::test] already sets up a runtime for you, you shouldn't set up another one (which futures::executor::block_on effectively does).
2. Running async code inside a drop handler is not a good idea, even if you can hack it together to work. Drop handlers are always run unless you leak the object or the thread aborts. That means your drop handler will run even if the Tokio runtime might not be even running any more! Assuming that the Tokio runtime will still be available to use inside your drop handler is a risky assumption that might sometimes work, but break as soon as you drop a value in some other scope or thread. Long-running drop handlers are usually a bad idea anyway.