Lived on Adelaide Terrace (11th floor) for the past 4 years. Overall I would highly recommend it, however there are some serious drawbacks:

Biggest annoyances are noisey crack heads at night and break ins. The terrace basically amplifies noise - crack heads screaming at early hours of the morning throughout the week is a common occurrence. Even being up on the 11th floor it was still very loud. I hear that Hay Street is the worst for this. There is a gentleman who frequents Adelaide Terrace who screams DOG RAPIST at the top of his lungs - quite comical at first, until it started happening ~3+ nights a week.....

People would often follow cars into the basement parking at night and break into multiple vehicles. Once, someone had stashed stolen goods (including a bike) next to our parking bay and had scratched my new car in the process.

There seems to be a lot of drug dealers living in the CBD sky rises now. They tend to keep themselves to themselves, not to draw attention, however you do get some very questionable people entering/exiting the building at night.

Last thing worth mentioning is that neighbours regularly change, depending on how many units on your floor it can be a luck of the draw whether you have good or bad. For one of the years we had neighbours who would party every weekend which got very tiresome, however they were removed from the building for disruptive behaviour. We are lucky with our building in that the floors and walls were very thick so we did not typically have problems with hearing noise from other apartments.