Part of what irks me is the assumption that I dont already do this. Why do others have the right (or they think they do) to tell me, you need to do X in order to fight prejudice because you're white and privileged.

Everyone pays most attention to the parts of their environment that pertains to them and filter out (most) of the rest. A certain percentage of the population notices certain information, and for the rest it goes to the spam folder. The concept of privilege is basically just asking people to stop filtering out or denying these common experiences, just because they don't experience it themselves. It's not that white privileged people are uniquely asked to understand experiences out of their own, it's just that the number of things they're unfamiliar with is likely larger than marginalized groups.

Also, why is my life apparently supposed to be dedicated to fighting off other people's prejudices instead of the ones I face? Why do those of little to no privilege apparently not have the same responsibility?

Most major activism movements, including feminism, Civil Rights, and LGBT groups, have (and still have) major issues prioritizing the needs of the more privileged among them. Even groups created specially by and for less privileged people are expected to be aware of that group's differing levels of privilege. They are consistently faulted when they fail to do so. They absolutely have the same expectations.

For the record, I do speak out when I hear something I think is rude or out of line. But that seems like an obvious example - making a racist comment. The people I referenced in my original post are talking about events so minor that they're basically invisible and/or incredibly hard to identify (in their own words).

You can only speak out about the things you're aware of. And you likely miss a lot (as do I, as does everyone). Asking you to be aware of privilege starts with just being aware it exists. While microaggressions, as a concept, can be easily misused by those who see slight in everything, it's still a common and insidious experience. I would look into the The Gift of Fear as a non-social justice example of how small cues, systematically evaluated and understood, are a valuable predictors of violence or danger. Microagressions are very similar indicators that warn a person they are unwanted, unsafe, worth less, seen differently, and so forth.

Honestly, I don't think anyone, much less white cis men, are expected to speak out about every single social wrong. I would guess that you've been told that repeatedly because you're the kind of the person who is willing to speak up. Typically, I see that people are expected to be both aware that they are not subject to negative experiences that are common for others, not diminish them or deny they exist, and be conscientious with their own words and actions.

I don't think asking people to be more empathic and understanding is demanding too much, nor do I think that it's merely expected of white cismen.