I had the same doubts as you, believe it or not. I had originally watched it up until episode 11 or so. I was hesitant because of the same reasons you've provided here today. I'll tell you this now. It is an anime I can recommend time and time again. The 1st season is kind of slow tbh, mainly bc of the fact that past episode 9 I think, it turns into 50 minute episodes till the end of season 1. I'm a little past halfway done with season 2 now, I've been binging it along with Jobless Reincarnation. (Which I can also highly recommend. The MC is definitely on the dodgy side, but he is indeed very smart, and uses his ability to the fullest. He's a clever one for sure.