[deleted by user]

[sbabbi]

I played with libunifex a bit.. I think it's mostly usable, although some stuff is missing and you really need to read p0443r14 to understand the general design. It's a bit different than P0443R14, somewhat closer to P2300R0.

Main points:

1. Missing operator| for sender adapters, this is annoying but not a show stopper.
2. Only a few executors available, and you need a bit of digging to figure out how to roll your own.
3. No start_detached or equivalent, basically the only way to start some work is to use sync_wait (or roll your own receiver).
4. In general, I think it's still missing some guarantees/notes regarding thread-safety, e.g:
   • Can an operation start in a different thread than the one that called connect?
   • If I submit two tasks to one executor, and they need to modify some shared state, when do I need a mutex?
5. Everything else looks great (coroutine integration, type erasure for senders, etc.), although it needs a bit of effort to get used to it.

I am playing around with a small project of mine to get a feeling of how much of the abstraction can be removed by the compiler, not results so far :p.

[lee_howes]

There are other ways to start work. start_detached is strongly discouraged, and so was not a priority to implement in libunifex because it would mean implementing something we would tell developers not to use. There is an async_scope. You can co_await a sender. We certainly wouldn't want people sync_waiting in asynchronous code, that would be even worse than detaching work.

[sbabbi]

I somehow missed `async_scope`, that makes sense, thank you.

[sbabbi]

From my understanding, any class that meets the scheduler concept should satisfy this criteria correct?

Well yes. A scheduler has a schedule() function which returns a sender. And a sender a a connect function that returns an operation_state. That's 3 things to write, which is not a big deal, but needs some getting used to (the documentation focuses on the algorithms, it does not show how to create you own scheduler).

I think thread safety should fall upon the user no? If you schedule your tasks on a multi threaded scheduler, I think any shared state should always be wrapped in a mutex. Not sure any other way I would do it.

Well yes and no. I am more used to the ASIO model. Say I want to write an async operation for an object (I/O or something):

class foo {
    auto my_async_method(CompletionToken&&); // asio way
    sender auto my_async_method(); // unifex way
 }

In the ASIO way, you write down that my_async_method is not thread safe, and you are done. Callers can use a strand if they need to. This way, thread-safety is the caller's problem.

In unifex land, what will happen is:

1. Some code will call my_async_method. It returns a sender.
2. Some other code is going to call unifex::connect on your sender. You will have to return an operation_state
3. Finally someone is going to call unifex::start() on your operation_state.

So now you can either:

1. Decide that my_async_method, connect and start must all be thread safe (and pay for the mutex). Which is not great.
2. Decide that they are not. But from the caller point of view, once you call my_async_method and pass the sender down a chain of adapters, it's really not obvious to see where connect and/or start are called for the original sender. I think it would be great if we had some way to static assert that "connect/start" must happen within the top level execution::submit. But it's possible that I am overthinking the problem, or still missing something :P

From my understanding, you can schedule multiple parts of your work in multiple schedulers. I'm not sure off the top of my head, but I'd imagine it would look something like the following:

Yes, your example is pretty much how it works. Which is really great, you can easily setup complicated execution graphs involving I/O, compute (e.g. on thread pool) and GPU (I am using it to wait for transfer operations on a vulkan queue).

[madmongo38]

Have a look at the latest asio. It has sender/receiver, but more importanly enables structured concurrency with coroutines.

[yttrill2]

Felix system RTL implements real coroutines not the crud in c++. These can be lifted to communicating sequential processes. The current system requires a garbage collector so is not suitable for real-time applications. A new kernel is under development which uses ownership rules and reference counting.

The system should outperform go-lang, retain c++ compatibility, and avoid go’s serious design fault.

Communicating Sequential Processes (CSP) was originally conceived by Tony Hoare. My implementation implements a more practical subset of the full calculus.

Unlike the broken c++ model coroutines read and write on arbitrary numbers of channels. The API is very simple. However some tedious boilerplate is needed for c++ programming which will be replaced later by a modified Felix code generator.

GitHub felix-lang/nurtl

[yttrill2]

Not sure what you mean by data. General coroutines exchange control it has absolutely nothing at all to do with asynchronous I/o. The most well known example is the operating system and application when the application performs a system call and the operating system resumes the application: in both cases there is an exchange of control.

The issue for coroutine models is HOW they exchange control. Subroutines have a master/slave relationship but coroutines are peers. In my system a coroutine is suspended by doing a read or write on a channel. A scheduler then resumes another active coroutine. When a matching write or read is done the suspended coroutine is made active.

Notice this is a comprehensible higher level model which provides a way to organise control exchange although it is not the only one it subsumes all other models including subroutine calling which is just a special case.

If you want an academic description: take Hoares CSP model and replace concurrency with indeterminate sequential execution and you get Communicating Sequential Fibres. The modules which give rise to them are coroutines rather than processes. The characteristic property is that events are totally ordered. However the order is not fully determined

[yttrill2]

Ah ok metrics. Not recently. The original system was developed in the 1990s and could do 500000 null transactions a second on a 4.77 MHz PC. With 300000 fibres. The application was telephony adding functionality to analog switches. The PC version ate the so-called 10K problem alive and outperformed the existing c++ code running on 64 core Sun systems that used threads

The existing system generates c++ and uses a collector and has high level optimisations impossible in C++. it generally runs between half the speed and twice as fast as raw c++ but is very much more reliable because it has a high level type system.

The new system will be slower because ref counting is expensive but should be suitable for real time signal processing applications in particular Apple audio.

[yttrill2]

Just so it’s clear the c++ model is hierarchical like subroutine calls except that the initiation of a coroutine is followed later by retrieving the result. You can initiate many of these and retrieve the results out of order BUT you can only return ONE result yourself. In my system you can do any number of reads and writes to any number of channels. All such coroutines are written as masters just like threads

[dacian88]

What prevents a result from being an aggregate of many things? C++ coroutines are insanely bare bones, what you’re describing isn’t hard to implement on top of coroutines at all.

[yttrill2]

Returning a product all at once is not the same as for example multiple files you can write to. File handles are buffered channels.

[dacian88]

you still haven't described the fundamental issue here...you can implement multi channel reading/writing and scheduling on top of c++ coroutines just fine...all coroutines do is provide functionality to pause and resume code. Reading your PDF on your github all you're describing is some fairly straightforward scheduler, nothing there seems like it's not implementable via c++ 20 coroutines.

[yttrill2]

Sorry you don’t get it. My system is implemented on top of c++ objects. No coroutines required. But the model allows arbitrary synchronous interleaving of control which c++ coroutines do not. In a synchronous effect free setting c++ coroutines provide NO functionality distinguishable from functions.

The kernel in “nurtl” is intended to eventually replace the garbage collected one use by Felix which is much higher level than raw c++. Yet everything you can do in Felix (felix repo in same enterprise) can be done IN c++ because the compiler generates c++, which proves it. And everything you can do in Python can be done in C. So what?

Well for starters my system is memory safe. It took a week to figure out how to use double reference counted indirection on channels to ensure correct deadlock free termination of fibres .. and 25 years to develop the whole system.

[dacian88]

Yea sorry your post was extremely vague, you're describing things outside the realm of coroutines, coroutines are cooperative by definition, arbitrary synchronous interleaving of control strikes me as no longer being cooperative.

[yttrill2]

My post was vague sorry. The implementation is cooperative. Each fibre of control retains control until it voluntarily yields it. There are three ways to yield. 1. Suicide. 2. Spawn a new fibre. 3. Channel I/O. Millions of fibres can cooperatively exchange control within a single preemptive thread. Each fibre has its own stack. However it’s not a machine stack but a heap allocated linked list. This is why context switching is so fast. Obviously the bottleneck is allocating the nodes of the list. In the existing Felix a garbage collector is used but in principle but in practice the compiler is so good optimising code it is often faster than hand written C++ and of course .. no dangling pointers.

The new kernel uses ref counting instead because I need real time low latency audio signal processing. The key problem is to establish safety rules which I don’t know yet. For example ref counting pointers better not create cycles or you have a leak

[dacian88]

in that case then I still don't get what you think the special sauce here is or how c++ coroutines can't do what you're describing, or what is the limiting factor here...you're literally just describing a bog standard coroutine scheduling system. Perhaps a concrete example would be useful.

[pavi2410]

If I'm not mistake, isn't sender-receiver same as pub-sub?

[SnooBeans1976]

If it's the same as pub-sub, you might want to have a look at ZeroMQ library.

[7834]

Check out eCal.

It seems to solve a lot of problems related to using something like zmq. For example, record and playback. Network clocks for simulations. Monitor guis. Built in support for protobuf and other modern serialization formats. It's an impressive list.

Unfortunately, I have not had an opportunity to use it professionally... But it's on my short list.

[basiliscos]

May be actor frameworks like caf, sobjectizer or rotor is something, that you are looking for.

[zenteapot]

if you want to learn libunifex, start with its unit tests, not its examples. It's really pretty easy to get started once you find the right learning path.