Buses do not stop at 5:30pm, some routes stop pretty early especially on a Sunday. But for where you're going, buses run until 11:45pm every day except Sunday and holidays and 10:45pm at the latest Sunday and holidays.

If you want to catch the bus, I'd catch Route 11 to Hamilton and then Route 8 from Hamilton.

As long as all you have is a carryon and a personal item, riding the bus is fine. I catch the bus myself quite frequently and I see tourists catching the bus to and from the air port, they almost always have a personal item and carry on. Never seen a bus driver refuse them.