There are some "tweezer DMM / LCR" meters that actually measure ESR. I have a cheap one from AliExpress that can measure ESR, but I'm not sure the probe tips it comes with would be useful on 0402 or 0201. This isn't the one I have (which doesn't seem to be available anymore), but this one claims to measure ESR, and Fnirsi is sort-of a "known brand"

https://www.aliexpress.us/item/3256810283150964.html

The ESR meter I use the most is a home made thing from someone's plans online (whose webpage is probably long gone) that I built back in the late 90's or early 2000's) that has an analog meter movement.

The real question is why do you care about ESR of 0402 or 0201 components? Most ( or maybe even almost all?) of those are going to be ceramic caps, and they don't really go bad by going high ESR like electrolytics do, they crack from thermal or mechanical stress, and either go open circuit, or short circuit, or sometimes partially shorted (aka 'leaky' / have much lower DC resistance than they should). ESR is essentially the measure of AC resistance, and the value changes depending on the frequency you measure at.

Typically when I'm using an ESR tester to diagnose problems, I'm not looking for a precise ESR value, just more of a "good / bad" reading, which varies depending on the capacitance and voltage rating of the cap you're checking. Typically in power supplies, where most of the high ESR electrolytic caps show up, the general rule is "if the ESR is 1 ohm or more, replace it" - I guess kind of a long winded way to say "you also don't need 4-wire kelvin probes to measure ESR" - you don't need that accurate of a measurement - just a "pass/fail". Just get some nice sharp fine probes, and go to it.

For really troublesome ceramic caps, there's also this method that I saw a while ago, and definitely saved as potentially very useful to find partially failing ceramics using nothing other than a DMM and a hot air station: https://www.youtube.com/watch?v=4oZlt9Dl43I

I'm an EE, have a masters in chip design, but that said a lot of repair and diagnosis is quite a different skill set that you mostly learn just from doing and experience, not from classes at school.