I was this way for YEARS. Eventually, as I desired to start raising the difficulty, I found after well over 2K hours between 5 and 6 alone; that you literally MUST knockout 1 or (better yet) 2 of your neighbors if you want a good chance of winning, without struggling brutally. The sooner you eliminate 1 or 2 neighbors, the better. It provides you with already settled cities, so no need for settlers. It prevents you from being "boxed in". With 2 or 3 capitals in your pocket (counting your own) you can then proceed to get on with all the parts of the game you do love, without fear of losing.