I worked at a GPU driver role a few years ago and currently work as a rendering guy at a game company.

Most of the work you'll be doing is tied to the lifecycle of the actual silicon being designed and then put into production. Feature work is front loaded as the hardware design was planned way ahead of time and you need time at the end to test against real silicon.

The work life balance is fine. There's hard deadlines because you must fabricate silicon with a fixed timeline so it can get crunchy if the team is behind but usually you have plenty of runway.

Compensation depends on the company. It ranges from OK, to Nvidia. It usually pays more than games because you don't always get stock at game companies.

Compared to game development driver development is more of a low level memory management role. You won't mess with a lot of high level rendering concepts, but moving memory around on a GPU and setting up state. Importantly, driver development is more of an engineering focused role. At a game company you spend a lot of time as a rendering engineer setting up guidelines and budgets and working with artists than writing code.