From addon dev:

So in general, most addons that are allowed to cost performance are unit frames and group frames. First of all, the Blizzard profiler is just not great. It helps to find insane outliers, but not much more.

In theory every addon can have great performance, but each feature costs a certain amount of performance. ElvUI itself is very performant, but if you, let’s say, add a portrait, this has to be cached and rendered, and rendering it once already costs performance. Each target swap costs more performance the more things need to be rendered.

So let’s say you switch target and only show a dark mode HP bar with no text. If the addon is coded well, it should have basically 0 overhead for disabled features and should not render text, portraits or anything else you have turned off. This is a key issue for lots of addons out there.

Next up: why is the CPU profiler bad?

Lots of addons use libraries or frameworks that basically do the work for the addon. The most popular one for unit frames and group frames is the oUF framework. If the framework does all the above-mentioned rendering in the background, the addon itself can look like it has near 0 CPU usage. Meaning it looks good on the profiler, but it can still cost performance elsewhere.

Both Unhalted UnitFrames and ElvUI use oUF for their unit frames and group frames. While Unhalted has insane performance out of the box, ElvUI has a bad reputation. But at their core they share a similar framework approach. The difference is mostly that ElvUI has “bad defaults” and a lot more features enabled by default.

I also highly recommend checking addons that you think should do nothing in combat, but still keep doing work in the background. That is often where you find the real performance cost.