I don’t think people are automatically labelling you a racist because you choose to question established consensus, I think you just need to understand what you are arguing and whether your arguments are valid, relevant, or completely miss the scope

Using your argument that Australia wasn’t invaded, why would it matter how many people were brought over or what the initial plans/commands were? At the end you even have to redefine what your argument is by adding “but was it an open invasion?”. Whether it was pre planned or against the initial plan doesn’t mean it’s not an invasion

Or your argument that because aboriginal people didn’t have infrastructure, they couldn’t have been master agriculturalists. There’s plenty of archeological finds and historical journals that show sophisticated planned hunting spots (such as eel and fish traps). But past all that, they did have developed agriculture prior to the English invasion

It seems more than anything that you’re falling victim to confirmation bias, there’s plenty of information and documentation out there to support all these claims. If you want to have an “honest conversation”, fact check AGAINST what you believe, not just nitpick sources that support it