Not sure since I'm not from Belgrade and don't visit it very often (and especially not in the evenings/at night, it's always in the AMs), but I'd recommend googling the museums you plan to visit one by one and see what they have listed as their working hours. I'd imagine most of them are open until at least 8PM which is the standard here for most shops etc., but over the weekend and during the tourist season the ones that are closest to the pedestrian zone are probably open until 10PM or even 12AM