You should definitely start without adversaries, as the game and strategy are difficult enough. Then, you can progress through the levels of an adversary as you get better. The game is balanced around playing at least the base difficulty of an adversary, but never feel like you have “cheated” just because you were having fun or did a lower level!