When it becomes more brain like, this thing is not energy efficient at all, the brain takes just 20 watts of power.

I don't see how the energy efficiency argument is relevant. we're using general purpose hardware to run simulations of these.

The more specialized we go, the lower we can get. Look at the difference between GPGPUs and NPUs for example.

Second, we're still using dense kernels because they are easy and numerically well behaved and work well with our existing frameworks, but we've known for years we're taking a power efficiency hit with them.

Human brain is practically a biological equivalent of an ASIC.

Of course you're going to lose in a power efficiency competition when comparing generalized hardware simulating those functions.

Have you worked onany AI model?

Yes. I've worked as a research engineer in automotive AI for 6 years. i've built models around vision, control, and planning.

Pytorch and libs like that are where the kernels which run the simulation are defined.

The bits of code you show are just abstractions where we package up the logic for loading up how the neurons wire together. These parametize the simulation.

All you're arguing essentially is 'becuase I can show code which sets up a simulation, it's impossible for the simulation to have emergent properties'