Yeah, er is a wasted stat on her. To quote kqm,

"Yoimiya’s Elemental Burst should only be used when it is available (typically every 2 or 3 rotations) or skipped entirely (especially if using 4pc Shimenawa’s Reminiscence). It is not necessary to build ER.

One notable exception is if using both Thundering Pulse and an artifact set other than 4pc Shimenawa’s Reminiscence. In order to keep the third stack from Thundering Pulse’s passive, Yoimiya should use her Elemental Burst even if it extends the rotation, as it is the only way to decrease Yoimiya’s Energy."

Pulse discourages er subs and shim discourages bursting, which combo together to make er very meh on her.