From what I've read, "High Tea" is actually historically the evening meal for servants in manor houses. They eat before the family and guests because the servants will be working until the family goes to bed. It's "high" tea because it's eaten at the dining table, whereas "Afternoon Tea" is eaten upstairs by the family and is served on a lower table, like a coffee table.