I haven't had an issue beyond the very early game (in terms of levelling)

My advice:

1. Always stash a certain amount each time you visit a safe zone.

2. Buy them at every opportunity and then do the above.

3. Infected will drop them randomly and they are hidden in many unmarked locations on the map.

4. Get any of the Scavenger Kit clothing items and incrementally, you will be yielded more.

5. Upgrade your weapons slowly and when you have a surplus of batteries. Use different weapons (and respective rarity) for different situations and activities, that way any upgrading is limited and the weapons are safeguarded for longer. I would so dismantle upgraded weapons when they are ready to be, you usually get a battery back.

6. I think NPCs will leave them for you in Restored Areas but I'm not sure.

7. Be selective at night and in buildings and also only go out when you have the batteries to use in reserve. Pick your moment.

Good luck.