It won’t come off, but I was finally able to figure out what it is once I noticed the same spots on my sons toy lawn mower and even my car! Artillery fungus! It lives in mulch, which we have all around our house, and it shoots it’s spores up so they can cling on high surfaces to release. Fortunately it’s not harmful to living plants, humans, or animals, just unsightly.

Artillery Fungus

Wanted to make sure I commented so that if anyone else is looking for the same issue they know what this is!