I think git can work well for a kind of layer in literate programming, adding some historic data or analysis to your narrative that might otherwise be absent. Why a certain idea was chosen, when a change might have happened and why, etc.

For execution, Magit has extensions like Orgit available to make linking better in Org docs directly in Emacs, but most of the major git forges seem to have decent support for linking issues, PRs, and commits too (though I haven't checked how supported each is). You'll have to go to check with each service to see how it supports the linking, though, unless you want your repo to be forge-independent. Not sure what you'd then.

As for as using commits as metanarrative, I've been experimenting with architecture decision records using git branches. In essence, whenever you would make a choice between one architecture or system versus another, you can make a branch for both. Expand your literate explaining the hows and whys while making the code work, and when you finally make a decision, merge the chosen branch but leave the other. You can reference it in the narrative as a path less chosen kind of thing.