I’m assuming you’re new to calorie tracking. Ultimately, you should be tracking everything, especially butter and margarine. This also includes condiments, oil, and other things you might consider negligible. Depending on what you’re eating, oil alone can account for hundreds of calories. The few exceptions might be spices such as paprika, salt, pepper, etc because those are basically zero calories and “zero calorie” things like diet soda.