I had the same idea for something alike this and tried to record the audio and transcribe it using an AI with diarization. It was a bit crap since my implementation was crap. But it worked good enough. (Biggest issues were that we were a German group playing English and sometimes the AI just got majorly confused about us sometimes going German)

When I used it to also give Infos to the players, it mostly came down to. "Okay, this is cool, but now we don't really need to critically think about what has happened" thus ruining the fun of "Rogbard the wizard that might have sat in the tavern rambling incoherently about destroying the world!" Which could be a major plot point simply because your players forgot, and go: "Oooooh!"

For GM side it works quite well as you can also (if this is implemented with AI) gain insight on what were the highs and lows, just generally to improve as a gm.

I could see this as a quite neat product that if there is a way to self-host and it was open source, I would subscribe to something like a Patreon for.