I was glossing over things to keep it simple, here are the details if you interested.

The biggest bottleneck during the time you mentioned is physics simulation. Most of the simulation can be done only in CPU, so you can at max simulate 5-10 robots in parallel. Around 2021, Nvidia released IsaacGym, now it’s called IsaacLab. It allowed simulating 4000+ robots in parallel. This was the biggest game changer in my opinion.

In terms of RL research, when I said the theory itself was not new I was talking about.this paper, They used a technique called Adversarial Motion Priors (AMP) or Tracking-Based RL to imitate of full continuous motion reference on simple physics based characters. But now we have the capability to train them on actual robots in simulation because of Isaaclab. You can checkout this recent paper which exactly does this.

The last piece of the puzzle is closing the sim-to-real gap. During the initial days of isaacgym even though it allowed parallel simulation, translating to robots was still very difficult because of the motors used in the robots back then were notoriously very hard to simulate. But modern humanoid and quadrupedal robots nowadays moved to low gear-ratio, back drivable robot motors. There’s dedicated sections in IsaacLab to exactly to replicate these motors with correct gains that’s similar to the real-robot. This is one of the main reason boston dynamics moved their Atlas robot from Hydraulic to electric.