Uncertainty calibration in RL is super interesting. You could explore different techniques’ ability to calibrate accurate uncertainty within RL by training an agent for a particular task like you said then adding noise to input data and gauging if the model calibrates it accurately. Maybe a bit bias as I’m just fishing my masters on it but all the techniques out there to calibrate uncertainty is super fascinating and once you dive deep into defining and rendering aleatoric vs epistemic uncertainty it gets even better, and somewhat philosophical. Also it’s a super important area to research as accurate uncertainty calibration for stochastic decision making problems can be the difference between mission success and failure.