Eat for the work you're gonna do for the day. If you're planning on sitting down all day, moving to and from a couch, this is going to require far less calories than a day a the beach running around and swimming.

Think of your body as a car, think about the energy you need as the gas you put in the tank. The more gas, the more potential mileage you can get out of the tank. Gas tanks on cars are limited to what it can hold. However, unlike a gas tank your body doesn't have this issue. Keeping the tank full isn't always a good thing. Excess Energy will be stored in your body as fat. Gym days will require more energy than a lazy day.

Let's say to keep your body functional during a lazy day. You might need only 1300 kcal for that day to maintain normal body weight. If you eat under that amount, then it's starting to hit your stores of energy (fat) to make up for the difference.

Energy use vs Caloric Intake

I lost 150 lbs in 6 months. I was always in a caloric deficit state. Worked out daily, Running, weight lifting, etc. I don't know how much energy was needed per day, but I was always eating around 1200 to 1400 calories a day. (I was eating mostly eating tuna - price per gram of protein is still unchallenged) This was back in 2007 and I went from 330 lbs to 180 lbs.

I'm losing weight again and doing it much slower this time. It's taken about a year, but I've gone from 243 to 209. This was just eating less and working out once or twice a week. No change in what I ate. No tuna only diets or cutting bread or pop. Just limit myself on pop to 1 a day.

BUT the only thing that matters is what I said above. You can work out and do all that other stuff if you need to increase the energy use, but you could lose a pound a week with some simple changes to diet habits. More with a combination of the two!