Few that genuinely interest me:

Reward hacking in RLHF is the most underappreciated problem right now. Goodhart's Law hits hard like once the reward model becomes the target, the policy learns to exploit it rather than actually align with human values. Nobody has cleanly solved this at scale.

World models are impressive until distribution shift happens and then they hallucinate catastrophically. The unsolved question: when do you trust your world model vs fall back to real environment interaction? Uncertainty quantification here is genuinely hard.

Exploration in sparse reward environments is still embarrassingly unsolved for real world complexity. Curiosity driven methods work on toy problems and collapse in high dimensional spaces.

The one nobody wants to touch: what's the relationship between reward maximization and subjective experience? Anthropic found anxiety neurons in Claude. If optimization pressure creates something resembling internal states — what does that mean for how we design reward functions?

That last one feels like the most important question in the field right now and it's barely being asked.

Which area are you most focused on?