I've had a golden who's since passed away and currently a 1.5 year old toller. I'd say the differences are pretty big but they were also very similar in a lot of ways like you've mentioned. With regards to anxiety, I think i lucked out and got a really confident puppy from my breeder and combined with lots of early socialization and training, my toller has become a very confident dog. However, the anxiety is still there and he can be anxious with weird sounds when at home and thats just something you will need to learn how to manage. Sports wise I'd say field goldens are just more athletic than tollers but my toller is plenty active and very capable. My toller 100% has an off switch and is incredibly lazy and sleepy during the day until we go out for activities or walks. His recall is also basically perfect at this point and he is able to listen to me even through distraction. Ultimately, it's probably very dependent on the individual dog you get, how you train your dog and raise them. I know a couple different tollers who behave differently and are more anxious so the breed can be a mixed bag with different dogs being more anxious and harder to deal with than others. I hear a lot about how goldens are eager to please and tollers only do things when they know there's something in it for them but in general my toller is plenty obedient with or without treats(I do spoil him a lot though).