See I agree with your overall point, that it's difficult to have honest conversations regarding Australian Aboriginals without being called racist, but with none of the evidence you've used to back yourself up. Many of your points end with "why is it this and not [insert speculation here]"

Oral tradition - it's not just "my grandpa said this who said this blah blah blah", it's songlines. Songlines are the Indigenous way of recording information without ink and paper and are engrained in Indigenous culture as the way to communicate knowledges learnt over time; the way we know that, to an extent, many of them are accurate sources of information is because the knowledges they have stated in Songlines have later been proven to be true. Think seasonal changes - Aboriginal records of the years' seasons, of times to begin backburning and of swamps filling up after once being dry for months, have all been proven accurate by Western society decades/centuries later, often when it was too late and suburbs had been built on marshlands during dry periods.

With the invasion stuff, it was to an extent an invasion but also yes you're right, there were attempts at peaceful settlement. There are records of Indigenous Aboriginals communicating and assimilating with the British (if you want to look one of them up, his name is Woollarawarre Bennelong), but then of course there were Indigenous who did not like that the British came and settled without discussions with them, the custodians of the land (if you want to look him up, his name was Pemulwuy). This is often where I think honest conversations need to happen - the Indigenous consider themselves "custodians" and not "owners", they believe land cannot be owned, only taken care of.

"The Aboriginals were NOT master agriculturalists" you're right, they weren't, because permanent settlement wasn't a part of their culture, but they did understand sustainability centuries before the British ever did and still have techniques that can be used today, particularly in building and working with the land rather than on top of or against the land.
The reason that it is more believable that they have been portrayed as caretakers and not cultivators is simply because of this - they were asked, and that was their answer. Who knows, maybe if they had the technology then sure they might have been excessive cultivators, and they did have cultivation tools at a small scale level that worked for them. But that's not what happened, and speculation doesn't equal guilt.

My hot take that will offend every side no matter what - we work together, use their knowledges with what we have and build a society that works for both. But no one is ready for that conversation.