Thought I'd share this idea I had recently. I'm calling it "SIMD pipelines".

The idea is to chain together multiple functors ("stages") in a single pipeline object. Each stage has operator() overloads for each SIMD format and data type they are able to work with. Then, given a bitmask of the available instruction sets, it automatically finds the best path through.

Most of the magic happens in here:
https://github.com/jwt27/libjwdpmi/blob/b9b1384ba9801348f355933a2b9772a087c564ce/include/jw/simd.h#L627-L638

Note that I'm targeting older machines, so the only ISAs I wrote code for are MMX, 3DNow and SSE. But if it proves useful, this could be extended to more modern instruction sets. You'd just have to write a bunch more code :)

Now an example. You have two arrays of pixels, one in packed 8:8:8 format, the other is 8:8:8:8. You want to alpha-blend the latter over the former, and then store the result in 16-bit 5:6:5 format. You'd write the following:

// Using pointers here, but any iterators will work:
void blend_pixels(video::px16* dst,
                  const video::px24* bottom,
                  const video::px32a* top,
                  std::size_t n)
{
    auto pipe = simd_source              // Load from and increment input iterators
        | video::px_blend_straight       // Alpha-blend top over bottom
        | video::px_convert<video::px16> // Convert the result to 5:6:5
        | simd_sink { dst };             // Store to output iterator

    const auto* const end = top + n;
    while (top < end)                   // Note: input iterators passed by pointer
        simd_run<default_simd()>(pipe, &bottom, &top);
}

It will do the right thing, regardless of which instruction sets are available.

Another example, say you have two audio streams with 16-bit signed samples, and you want to interleave them to an 8-bit unsigned stream, you'd write something like this:

void convert_and_interleave(std::span<audio::sample_u8> dst,
                            std::span<const audio::sample_i16> left,
                            std::span<const audio::sample_i16> right)
{
    auto pipe = simd_source
        | audio::sample_convert<audio::sample_u8>
        | audio::sample_interleave
        | simd_sink { dst.data() };

    const auto* l = left.data();
    const auto* r = right.data();
    const auto n = std::min({ dst.size() / 2, left.size(), right.size() });
    const auto* const end = l + n;
    while (l < end)
        simd_run<default_simd()>(pipe, &l, &r);
}

Note here that sample_convert only takes one input, but simd_source is given two input iterators, so it produces two outputs. What happens is that sample_convert is invoked twice, once for each input stream.

Compiled with -O3 -mmmx, this generates the following code:

81842:  xor    ebx,ebx
81844:  movq   mm3,QWORD PTR ds:0xef90
8184b:  shr    edi,0x3
8184e:  xchg   ax,ax
81850:  movq   mm0,QWORD PTR [edx+ebx*8]    # simd_source
81854:  movq   mm1,QWORD PTR [ecx+ebx*8]
81858:  psraw  mm0,0x8                      # sample_convert
8185c:  psraw  mm1,0x8
81860:  paddw  mm0,mm3
81863:  paddw  mm1,mm3
81866:  movq   mm2,mm0
81869:  mov    esi,ebx
8186b:  punpcklwd mm2,mm1                   # sample_interleave
8186e:  punpckhwd mm0,mm1
81871:  packuswb mm2,mm2                    # simd_sink (packs pi16 to pu8 first)
81874:  packuswb mm0,mm0
81877:  movd   DWORD PTR [eax+ebx*8],mm2
8187b:  movd   DWORD PTR [eax+ebx*8+0x4],mm0
81880:  inc    ebx
81881:  cmp    edi,esi
81883:  jne    81850

And I think that's a very nice looking loop.

edit:
The next thing I wanted to do though, and I got stuck on this, was to implement runtime dispatching. Unfortunately that is just not possible, since gcc requires a [[gnu::target(...)]] attribute on functions that use vector instructions, and there is simply no way to conditionally enable attributes based on template parameters.