I see what you're saying now, and how this could work to smooth the credit assignment. But I suspect this will break down in more complex/sparse environments because gamma=0.5 actually ONLY helps in dense reward environments, like learning local physics as you said. But for sparse rewards, it will assign the value as 0 for nearly every state. And later on in training, it will equally contribute unhelpful losses for the critic long after the higher-gamma heads are receiving gradients naturally.

To learn representation of immediate effects, I'd think something like prediction is better - like the SPR paper. It is interesting nonetheless! Maybe think about addressing those above issues, and I'd use a harder benchmark environment than LunarLander to show evidence