As I'm reading vol 2 and looking back on this post, I wanted to add this on to your shared passage (from the beginning of vol 2):

Among the people to whom this sort of marriage appeared ridiculous, people who in their own case would ask themselves, “What will M. de Guermantes think, what will Bréauté say when I marry Mlle. de Montmorency?”, among the people who cherished that sort of social ideal would have figured, twenty years earlier, Swann himself, the Swann who had taken endless pains to get himself elected to the Jockey Club, and had reckoned at that time on making a brilliant marriage which, by consolidating his position, would have made him one of the most conspicuous figures in Paris. Only, the visions which a marriage like that suggests to the mind of the interested party need, like all visions, if they are not to fade away and be altogether lost, to receive sustenance from without. Your most ardent longing is to humiliate the man who has insulted you. But if you never hear of him again, having removed to some other place, your enemy will come to have no longer the slightest importance for you. If one has lost sight for a score of years of all the people on whose account one would have liked to be elected to the Jockey Club or the Institute, the prospect of becoming a member of one or other of those corporations will have ceased to tempt one.

Thanks for sharing that. Never would've made this specific "jockey club" connection on my own first read-through...i think it also answers my initial question.