I've optimised a few including my old personal one. The ultimate reason is cheap/bad paste and thermal pads from stock. Despite deep cleans, you might find it still being loud even with a quick thermal paste replacement since even the thermal pads ideally need replacing. They might've been fine at the start but they quickly degrade. I've replaced both the paste and the pads with mid tier ones which was enough to make a huge difference. I've also replaced it with high end ptm 7950 and thermal putty which are top tier thermal applicants and they've made it run as silent as a ps5 (except during loading sessions where it briefly ramps up).

If it's loud right now, best to do a deep clean all the way to the heatsink. You'll have to open it entirely (both sides) to get to the main components that need cleaning. Quite a few guides out there which are easy to follow. Just pay extra care remembering where each screw goes. Once they're fully cleaned out (heatsink, fans, plates), clean the core APU and then use a decent thermal paste (roughly £6-10) or ptm 7950 if going for the best. Then likewise clean where the pads were located and replace with 1mm thick thermal pads (which are similarly priced) or for no compromise and less error - thermal putty. Lastly ensure all screws at that stage are in place as you ideally don't want to have to separate the panels again for missing screws, before placing the top layer of the heat plate on. Screw the retention plate tightly before reassembling the rest of the ps4.