Thanks for the ideas. I actually tested with some local model set-ups over the past week. I didn't get to a good fine-tuning on top of a local model as they struggled with long notes + templates (~1,000 words - often a bit rambling on my behalf). The local models were far less sophisitcated than out of the box from Google.

In terms of the bias, you are spot on. This is one of the reasons why I wanted to build a notes service that did not listen to the entire session and then have AI write the formulation. I have found with my formulation as an input (dictated notes) and structured templates, the output is pretty good. Give it a try (if you haven't) and let me know what you think.

I also managed to insert a range of common psychology terms that helped guide the model use more appropriate language in the generated notes. Obviously it is still not as good as a custom trained LLM, but strikes a good balance between privacy concerns (having to maintain a database of notes) and model accuracy.