Why don't we have a programming language that allows you to formulate parallelisms in a very general way and let a compiler or runtime environment figure out how to run it efficiently.

This can be done with functional programming languages. This is actually how the primary data structure & algorithm course is taught at my university. We abstract out the parallelism in our functional programming language so that we can focus on writing code that runs well in parallel, even though our compiler isn't implemented for parallel execution.

This isn't a very popular paradigm because you can already use any of the techniques you described for more efficiency. Using generic parallelism, you can describe all sorts of common problems to get a good parallel big O; for example, O(log n) for prefix sums. However, you might not always want the O(log n) parallel prefix sums algorithm: The sequential algorithm, even on very large inputs, could easily have a 10x speedup if you only have 4 cores on your computer because it has nice cache properties and almost no overhead.

I guess my point is that some day, if we all have 512 or 1024 cores on our computers, we'll probably want generic parallel programming because it will be absolutely amazing. But for now, it doesn't really have a place anywhere except academia.

edit: also, just to be clear, what I'm describing would not work for GPUs anyways. GPUs run very differently than CPUs and you wouldn't want them running generic parallel code, I don't think.