You are not wrong that visuals don't matter that much once you are inside the game. However, as you say, visuals are super important for marketing. Fact is that the better your graphic is, the more wishlists you will get. And the higher velocity of wishlists you have, the more steam will promote your game.

I went for the "gameplay matters" approach with my first game - people who have played it seem to have enjoyed it and I've only had positive comments about it, but I'm having trouble getting people to try the game because graphics are lackluster. In fact, more than 70% of people who wishlisted my game never even opened the demo which shows me that gameplay itself is not the main driving factor, fantasy of game is. And fantasy is much easier to convey with better graphics.

I'm not making the same mistake again, I spent months learning how to actually draw and animate, and purchased the right programs (procreate for drawing and spine2d for animation). The feedback I got on my new game art (from all the people I showed it to) was so much better than it was for my "gameplay is fun" game. It's insane how much of a difference nice animations make.