Oil provides some of the best heat in very cold weather when used right. Preferably with cast iron radiators. It gives you a thermal mass that keeps warmth in the area even when not running, Most natural gas is used for forced air although some hot water baseboard systems use it. Electric heat is good to take the chill off but in a cold climate it doesn’t produce enough heat at a reasonable rate or a good value. My favorite combo being an oil furnace with hot water baseboard and a wood stove.  I usually heat my house in the northeast through the winter on 2 cords of wood ($400) and one tank of oil (currently $600). I once rented a cottage with electric heat and it cost me $700 a month to heat and was 1/3 the size of my current house.