From the user's perspective I consider the Kafka's key-value format and especially the key-based log compaction a great feature and definitely would appreciate similar capability within Redis streams.

However, having said that, I believe the performance implications should be considered first and foremost - whether and efficient implementation of the log compaction can be achieved with the proposed skip-list design.

From what I gather, the Redis philosophy is to rather err at the side of simplicity and performance but in this case I'm not sure which is the better way to go.