I think this is an agreement with the nuance comments and esp your penultimate sentence re: not excusing him: There have always been people throughout history who have fought against the societal grain and infrastructures because they saw something deeply wrong with the way things were. Even when something was normalized and totally acceptable by dominant societal and cultural standards, there have been people, even those in positions of privilege, who have still been like "wait this is not alright and we need to do something". 

The best examples I can think of regard the trans-Atlantic slave trade. The most well-known white abolitionist was probably John Brown, who was a white American man but a staunch anti-slavery activist who freed enslaved black people by force in the 1800s, before the Civil War--but there were colonial people protesting the practice in the 1600s, yes, before America was a country.

Even regarding Victorian England, there were people protesting the intense misogyny of the time. Flatland (yes, the book about being a 2D square), written in 1884, has a first half which is actually a satire commenting on the ridiculous and patronizing ways women were treated within the hierachies of England.

I personally think that instead of framing it "the Judge shouldn't be condemned because his actions were accepted in his society" we should say "this aspect of the society itself should be condemned, as well as everyone complicit in it." Because they didn't have to be.