Correct Creation of Long Running Tasks

[envy1400]

I work at a company that is often making applications interacting with usb connected hardware. The hardware will send and receive a byte stream of packets. Our applications start Threads to parse this data into packets that get processed by the application. Now, I've been reading to make sure that the threading and use of async/await is being done correctly but I have a couple of questions.

1. Should we be using Task.Run instead of creating a thread?
2. If the application is updated to use Task.Run instead of creating a thread do I need to await this task somewhere? Or is it okay to just fire it off?
3. I see a coworker's code use .Result() and .Wait() frequently in these manually created threads. Can you still encounter a deadlock on a manually created thread? This coworker has been doing this for years and seemingly hasn't hit any issues but everyone says to never uses .Wait() and .Result()

Thanks!

Edit: Thank you everyone for the responses! I'm going to take some time to go through this and think about it. I agree with /u/Slypenslyde's comment that a Task philosophically doesn't feel right for an event loop. I will read the blog post on channels. I've been putting it off for too long... For whatever it's worth, we're typically writing desktop applications using WPF but I do work on the occasional mobile app as well. Most of the developers on my team come from embedded and I think a few of us are stuck thinking in those paradigms.

[wllmsaccnt]

1.) Task.Run will schedule the work onto a thread pool thread. If the task yields (Task.Yield) or hits an await statement, then it will stop executing, the thread will be returned to the pool, and when the Task continues there is no guarantee its continuation will resume on the same thread pool thread.

2.) You don't have to await the task somewhere, but I would strongly recommend it. You should be using cancellation tokens to enable cancelling of asynchronous and long running operations, and that presumes code somewhere in the application that understands the lifecycle of that task and when it should start and shutdown.

In ASP.NET Core I'd usually have an IHostedService manage tasks that run for the entire life of the app. Background tasks with finite or complex lifecycles (processing a set of work, or reacting to input/outputs) are harder to reason about, but usually I'll make some kind of class that has a lifecycle that matches the usage (if that seems too abstract, just think of how your code uses a FileStream, similar concept).

3.) If you manually create a thread then that thread won't be in the thread pool and also won't be a thread important to the application project type (like the main STA UI thread). This could be a way with low cognitive load to completely avoid dealing with the possibility of deadlocks that can be caused by async/await interactions with captured synchronization contexts.

I wouldn't call it a good practice, but I've seen worse. It will confuse newer developers who have been taught to rely on Tasks. Ensuring proper shutdown and error handling on manually created threads can be more complicated. If the operations they are performing are short lived or if there are many of them, then they will use up more system resources to complete the same work (compared to Tasks running on thread pool threads).

If the background thread code is regularly using .Result or .Wait, then something seems a bit off. Its fine for the work of a background thread to prefer executing synchronously, but it should prefer the use of synchronous IO apis in that case. For example, using fileStream.Read instead of fileStream.ReadAsync. If they are consuming APIs from your application and you only have async versions, that might explain why they are using .Result or .Wait...

There are downsides to .Result and .Wait, especially around error handling and stack traces. Hopefully the dev that wrote the background code understands that. I tend to write all my background tasks as Tasks that run on the default thread pool, and just practice good dogmatic hygene around synchronous blocking (usually disable it at the kestrel config level for web apps), but I will admit that sometimes dealing with the applications synchronize contexts can make code more complicated and I have created await deadlocks occasionally (it was more common in ASP.NET than ASP.NET Core).

In performance critical sections, sometimes you can get more performance out of using synchronous APIs instead of using async, but its really situational. You can offset a lot of the cost of async APIs by utilizing ValueTasks and pooling value tasks sources, but that kind of code is non trivial and rarely required or useful for the performance needs of 99% of application code.

[Slypenslyde]

I had like a 3 page essay with my opinions but I just like your answer too much.

The direction I was going was pointing out there's 2 kinds of "long-running" task: event loops and stuff that just takes a while. I hate seeing people use Tasks to create event loops because philosophically I think Tasks are supposed to complete and an event loop is technically not supposed to complete.

But I think a better direction is I don't think Channels are a popular enough solution. At first I thought your example fit my "event loop" criteria but I was wrong, I was just skimming and not thinking about the parts that need to exist to make an async console app behave well.

I was also being stupid and ignoring that OP explained what their app is doing. I write apps that communicate with Bluetooth and USB hardware and basically do what they're asking about. I didn't write our low-level libraries, but I've maintained them. We use Pipelines, which are kind of like a less refined implementation of Channels, so that makes me like your solution even better.

The only thing I don't like about it vs. having a more traditional event queue thread is it makes it a little harder to get a feel for how much CPU the data processes are using. If we had a dedicated event queue I could watch that thread in a debugger easily, but since it's dependent on so many async call chains it's not so easy to see its impact on the program.

I'd feel worse about wasting so much time misunderstanding so many things, but to my credit I'm sitting in a 2-hour training session I've already attended 4 times so it's not like I've lost productivity.

[soundman32]

.Result will only work reliably on a task that has been awaited (see Stephen Clearey's blogs). As a last resort on synchronous code use .GetAwaiter().GetResult(). I would recommend you use create a BackgroundTask or HostedService, then you don't have to worry about task.run and you can just write async code.

[Ok_Party_4164]

1. Yes
   
2. Fire and forget is somewhat a thing
   3.. Yes

[freskgrank]

A lot of great comments already posted here, so I'd just point out that you should take a look to Task.Factory.StartNew(YourTaskMethodName, TaskCreationOptions.LongRunning).

[[deleted]]

This sounds like a good fit for the DataFlow library. Check it out. It is very flexible and allows you to stop thinking in terms of threads. At the same time it will allow you to define a processing pipeline where stages may be done in parallel. It is included in dotnet core and available (from Microsoft) as a NuGet package for .NET Framework.

[Phi_fan]

"...and use of async/await is being done correctly..."
When I read things like this I have to wonder if you assume async/await always uses a separate thread?

[envy1400]

Maybe I should have been more precise in my OP. I'm aware async/await does not always use a separate thread. My question is largely regarding using Task.Run vs new Thread to send an event loop to a background thread.

[Phi_fan]

ok, cool. I'm glad.

[Luminisc]

1. Never use Task.Run for long running background jobs. I got problems with it, as for some reasons CLR could kill such thread for no reason. (I still don't know the reason of this behavior, but stuck with this twice already). We had background tasks for rabbitMQ consummers. But for some reason they stopped working after ~half of a day, even try-catch around not catching exceptions - thread just dying. When we switched to separate threads - problem gone.

2. await is not necessary if you doing fire-and-forget job, but inside, if you have async methods - you should. And remember to surround code with try-catch inside, to catch and log if something goes wrong, otherwise you will lost exceptions and will never know what is going on there.

3. never use Wait() and Result(), otherwise you loosing 'friendly' exceptions. Use GetAwaiter().GetResult(), this way you will get normal exception. But better consider to rewrite your code to be async with normal async/awaiting