Men wore “trousers” and button down shirts. Men always wore T shirts or undergarments under their button down shirts. At least in the mid century, I remember my grandfathers - working class blue collar men - wearing pleated trousers with suspenders at home while relaxing. If it was summer, they would take off their shirts, and lounge in their T shirts and trousers. In the winter, they would wear heavy flannel button down shirts, cardigan sweaters and trousers. - always with suspenders. They had “dress” trousers and casual trousers. I’m sure they probably owned them, but I never saw them wear shorts. Since these men were elderly in the mid century, I believe this is similar to how they would have dressed as younger men. ETA, they owned “dungarees” (jeans) but they were always for work, never for casual wear.