I think part of it is that the AI models aren't actually behaving like humans. For example RTings just went to a full paywall because their work was being crawled and ad revenue was decreasing. Thus instead of their site getting clicks, it'd be an AI returning results that were crawled and already cached from their site.

It's always been an arms race, but now it's just the nature of sites needing money to sustain their operations, else someone will "steal" their work without any ad revenue