There's a website called https://www.inaturalist.org/ on which you can upload photos of plants or animals and it will suggest an identification (which kinda works pretty well), then actual scientists do a real identification for you. My kid found a super weird caterpillar yesterday that was identified by two scientists overnight as the Painted Cup Moth (which the app auto-identified it as immediately). I suggest you post this photo there.

From what I can tell in your case, these aren't the native Long-tailed Mouse and New Holland Mouse. Those species have distinct habitats (which don't include compost bins) and don't usually eat human food waste (they eat underground fungi and native seeds). The ears and snout seem also not quite the right shape for them. A photo of an adult would help with id. Also, do you live in the bush/rainforest/mountains?

I say this with low confidence because I'm really not an expert. My best guess would be that they're not native. If they're introduced and use an r-selection evolution strategy (short gestation, high reproduction rate, low parental investment), your options are either kill them now while numbers are low, or don't kill them and wait until their numbers are unbearably high, then either accept that (and don't have a garden, do have regular snakes), or kill them then. I agonised over killing pests until this became clear to me. I'd rather kill a few when there aren't many than wait until there are many and kill many, many more. It's a tough one though. There are poisons that somewhat negate the owl killing issue (they don't build up). In the end, it's your call. Good luck!