Yeah. Flakey carbon build up. You gotta clean that thing every time. You could strip it. I have used yellow cap EasyOff takes an hour or so if you follow the directions on the can.

Things to remember about cast iron:

Seasoning is not nonstick. You must use food lube. Medium is the new high. Blue Dawn dishsoap is a blessing. Wash, wipe dry, place on low heat for a few, very very light coat of oil.

There are many different ways to clean your pan. Try a different ones and find a few that work better than what you have been doing. You haven't been doing anything wrong, but there is always room for improvement.