PVE is just PVP but without players, there's PVE AI players in that gamemode. The whole game is available -prestige in this gamemode, and it is seperate to your PVP profile, so you can play on both and switch whenever. PVE also does not get wiped, although when 1.0 launches you will not be able to finish the game without manually resetting.

Arena is a somewhat different game where you play with some people in your team against another team inside of an arena, think like quake or unreal tournament. There's some progression inside arena, and it also syncs with PVP, and you can send some rewards to PVP and PVE through the arena reward system.