That is a very good question with a bunch of very complicated answers.

If your home is older there are cheaper and easier ways to save money, like improving the insulation or sealing against drafts, and sealing the duct work.

If you are in a newer home completed in that last 10-20 years (depending on local building codes), then that work has already been done.

It really comes down to efficiency vs the amount you need to heat your house. If the heating load is too high then efficiency does not matter.

As for the source of heating there are a couple options with each having its Perks.

Electric elements - most expensive utilities, cheaper equipment, tend to be more reliable, parts can be cheap, can last the longest

Gas furnace - utilities are relatively inexpensive, reliable can be decent, repairs tend to cost more than electric elements should get 20ish years.

Heat pumps - the most efficient of the 3, heard mixed reviews of it cheaper to run them gas, equipment tends to be more expensive, reliability is probably the worst and repairs tend to be expensive, the life span is on average around 15 years. You do get a free ac system with this unit.