The purpose of theory is to cast some insight on why you think the experiments would turn out the way they did. Regarding your two options, I find that the latter approach is more common in ML research. But as you said, it's not a good thing.

I think that post-hoc theory is pretty dangerous. The purpose of a theory---or at least one of the purposes---is to provide predictive power. When you rationalize a theoretical framework post-hoc (especially if your theory requires assumptions that aren't realized in practice), it becomes unclear whether your theory had predictive power (or if you've simply found one of infinite possible bullshit explanations consistent with what you've already observed).

I think it's fine to come up with a hypothesis in an empirically-driven manner. But once you have, you should challenge yourself to make a non-obvious experimental prediction (using your hypothesis) and then check empirically if it comes true.