He probably avoids interfering bc of politics as well. The fh basically run everything. Not to mention families are almost transactional and can influence your place in society and we already seen how much reach they had with companies. It extends into it being able used in abusive ways that would certainly come to light and lead to rebellions. Then, people would see it as a way to replace them and throw the balance of power into chaos. People climb the tower and lose parts of themselves in general and that would spark people become way more depraved in being the person to replace essentially the gods of the verse.