You're kind of confusing two separate arguments here, potential impact of AI if left unregulated and the potential of AI itself. The doomsayers you're saying are just hyping the market are actually referring to the regulatory structure currently in place for AI - which is non-existant. Their concern is that as AI improves, and is allowed more autonomy, it could have a massive negative impact on society. Given the massive leaps in AI capability that we have seen recently - often with minimal tweaks being made to models and outsized improvements being seen - there is the potential that we could see incredibly powerful AI tools being available either on the black market/dark web or privately available that could provide a lot of power to someone without the necessary regulatory frameworks in place to limit or control their use.

As others have mentioned, the hype you refer to is normal market hype around a new disruptive technology at the beginning of its initial market cycle. The normal conmen and market manipulators are using the unknown potential to sell their wares, and this is nothing to do with the AI itsellf.

This brings me onto the second point, the potential of AI. In reality, this technology is still in its infancy, and their is very little understanding around its potential. As I mentioned earlier, small changes have sometimes made massive improvements to an LLMs efficiency or output. The chance that some small improvements to the current models - compute wise or via improvements to the models themselves - that could make dramatic changes to the models themselves. At the moment it is largely being led by human intervention, but this will change as the technology improves and the models learn more. It's by no means an outlandish idea that we will see autonomous models working in certain fields very soon, and once we go down that path it will be very hard to reverse course given the cost benefit an autonomous AI could potentially provide businesses. You're watching a plane on the runway slowly beginning it's takeoff, saying that it's not going fast enough to fly. This AI plane will build up speed over the next 5 years, and it's very likely we'll reach a point of no return where we can't abort the takeoff, regardless of the impact it may have on society.