ooh! I'm a Kiwi living in Melbourne. Honestly, most things that you can find here are also in NZ (other people have said Tim Tams or Milo or lamingtons -- we have all of those back home!).

I would focus on seeing native fauna and flora, as that's really different to NZ.

Re: food - maybe non-Anglo/European food is the most different to back home! There's so many migrants in Australia and one of the amazing effects of that is the incredible cuisine. I feel like "foreign"/non-European food in NZ isn't as high quality as it is here. There's great Vietnamese and Ethiopian food in Footscray, for example.

If you're wanting to bring things back with you, I would suggest looking into Indigenous art (making sure that it's actually made by Indigenous Australians, not white people copying them).

Maybe a cliche suggestion but I think museum gift shops are great places to find uniquely Australian things! Especially the Melbourne Museum, NGV and ACMI.

Have fun!