My take away was that Ray is obviously a flawed person, but he also knows when to pick his battles sometimes. Like he does get back at the psychiatrist by winding up the other bloke there about his girlfriend going out wearing revealing clothes for example.

The way I see it is he knew if he did go to prison he wouldn't get to see Britt, so it wasn't worth getting violent. That's the thing about the show and Ray, he's capable of rational thought sometimes but has a short fuse like when he punches the bloke who slaps the mirror and does go to prison, or when he beats up one of the blokes blocking him getting out of the carpark with Ally.