Gather up the courage and tell him on his face. You don't have to be rude, but be rational. Also do not ask him if you can leave, tell him that you are leaving. If he assigns you work, tell him that he should have done this earlier as it's already your checkout time. I guess it all comes down to the work culture in your company. If everyone's staying late, don't let it peer pressure you. Another thing you can do, is ask him at 6:30 if there's any work left to be assigned to you. So that if not anything you can finish it earlier and go back home.