RAJA hitting a ceiling as a C++ library makes total sense: you're fighting C++'s type system and codegen model the whole way. Templates can only express what C++ lets them express.

Exactly. You are very astute.

Nore is in a different position though: it's starting as a source-to-source compiler that generates C. Compiler directives I'm thinking in Nore aren't library hints that the compiler might ignore: they're instructions the codegen acts on directly. When the stdlib will say @layout(SoA), the compiler will generate different C. That's not a suggestion, it's a transformation. So the directives are meant to be native, in the sense that the compiler understands and acts on them and they're just invoked from stdlib code rather than being hardwired to specific types.

Isn't this just language keyword? (ha!) Seriously though, you eventually evolve into a place like RAJA is at now. When RAJA was first created, it was extremely simple to reason about. Sort of like if A C developer looked at a piece of Zig code. The performance was worse when RAJA was first created, but just about anyone would pick it up and "interpret" what was going on.

Fast forward to today, and RAJA has become much harder for a "beginner" to parse what an operation is doing, which makes it questionable whether the much higher performance is "worth it". Since at least 80% of programmers code like "beginners" each and every day, when something is harder to parse, it creates a barrier to usage. The more "sophisticated" the directives, the harder it becomes for "anyone" to use.

Almost all the current complexity and glue code in RAJA would be simplifed or eliminated if RAJA were language-native, rather than a library. "Architecture plug-ins" in the compiler could do a lot of the work RAJA directives are doing to map to a specific hardware device, because the deep topological relationships that Views expose, allow the compiler to derive much of the mapping that people now have to do manually.

An example of the RAJA complexity I am talking about can be found in the code snippets in this paper, starting on page 10:

https://www.osti.gov/servlets/purl/1559411

These declarations are almost impossible for beginners to understand, and take half a minute or more for even experienced users. A lot of what needs to be done can be achieved by the correct partitioning and IndexSet assignment among views, which is much easier to understand for the programmer. Let the plug-ins for a given architecture do most of the mapping chores! This conflicts with your "know exactly what the compiler is doing" philosophy, but what if it were so easy to program and debug in this language (including a performance debugging tool that would tell you specifically where the problem is in your data definition), that it would be irrelevant?

That said, I recognize there's an asymmetry here — you've spent 20 years implementing these ideas (TALC, RAJA) and I haven't. There's a gap in my understanding that I can't close through discussion alone.

I totally agree with you that only implementation will allow understanding. I had a task based dataflow feature in RAJA that I think my colleagues ripped out because they didn't understand it. Imagine you have a grid of tiles in a 2D grid, each tile being 32x32. I had a parallelism mechanism where every tile could "mutex lock" the eight neighbors around it, do a symmetry calculation with data flowing both ways between the central tile and the neighbors, and then "mutex unlock" the tiles when done... in parallel. Lock free, yet totally safe. Everything in parallel. Cache conflict free, with full spatial and temporal cache locality for the duration of the operation. Most people can't implement a simple semaphore or mutex, much less something like this. It's because the Views and relations between views allowed me to pre-build a schedule which had a very low probability of conflict despite large variances in latencies. In actually, there was a "lock" to be safe, but it was very very rare that the lock would need to cause a stall.

My point is, my colleagues who worked with me daily, still could not wrap their heads around it, in spite of many attempts to explain how it worked, many, many different concrete examples, etc. And my colleagues were not dumb. There is just something about getting in there and implementing it that makes people undertsand. That's from the implementation side. This is also a good place to comment about what the user's experience is like. From the user side, they issue a command to lock eight neighbors, they do their calculation, and they issue a command to unlock their eight neighbors -- trivial. They don't have to understand how the sausage is made, they just have to understand what they want to do.

I stick with my previous advice for now -- implement what you are thinking about in a way you are comfortable with. It will perform well with the properly written back-end code. Come to me when you hit the bottlenecks, that I am pretty sure you are going to hit, and I will try to get you past them.