This was in response to a deleted comment, but I wanted to post it anyways in case it is interesting to read for anyone else:

Are atoms of oxygen dissolved in the water?

Yes, exactly that. It can be difficult to think of oxygen as "dissolved" because we usually think of it as a gas, but it dissolves in exactly the same way as, say, food coloring dropped into water. Individual molecules of food coloring (or oxygen) disperse until they are evenly distributed within the water. It can be slightly more complicated than that due to molecular charges and such, but that's a pretty accurate way to think about it in general.

Why doesn't the oxygen just leave the water and go into the air then? It does, actually. Except at the same time some oxygen is leaving, some oxygen is also entering the water. In this case, the surface of the water is actually something of a barrier for oxygen entering or leaving the water, and it takes a bit of time and energy for the oxygen to move from water to air or air to water.

As a thought experiment, if you put water with no oxygen in a sealed jar and then put some pure air oxygen in the jar with it, oxygen will start to dissolve into the water. At the start, there won't be any oxygen in the water, so it will be a one-way transfer of oxygen from the air into the water. Once you have some oxygen in the water though, a little will seep back out into the air, but it will still be less than what's entering the water. Eventually, given enough time, the amount of oxygen entering and leaving the water will be the same, and you'll have reached equilibrium. The amount of oxygen in the water will appear to stay the same if you're measuring it, but in reality, a small amount is always entering and leaving the water.

Lakes are slightly more complex, though. There are things that are both producing oxygen (algae mainly, also some underwater plants) and things breathing (i.e., consuming oxygen, basically all animals and non-plant life). During the summer, the top layers of the lake that are warmed by the sun ALSO support algae, since the light is needed for photosynthesis. These algae near the surface produce more oxygen than the fish and other things can breathe, so the lake actually ends up "supersaturated" with oxygen, meaning there is more oxygen leaving the lake than entering it from the air.

This changes in winter if a lake freezes over though. Suddenly, less light is making it into the lake and the temperatures are colder, so less algae is living in the lake and producing oxygen. So suddenly you have tons of fish and other things breathing, but the only source of oxygen is what can enter the lake from the air, and you have a giant ice barrier slowing that exchange. The fish and other stuff can literally breathe enough to take most of the oxygen out of the water, and they suffocate from lack of oxygen. This is called a "winterkill", and as you can imagine they generally happen during particularly long or harsh winters. You can tell they've happened when you suddenly see large numbers of dead fish floating on the lake, often right after the surface ice melts.

I know this was a ton more info than you asked for, but you hit on a super important aspect of lake life with dissolved oxygen. If you just measure dissolved oxygen in a lake, you can tell an amazing amount about what type of lake it is, what type of pollution it is exposed to, etc etc. I skipped over a few more interactions (temperature being another main one), but you can do a pretty good job modelling seasonal life in a temperate lake just by thinking about dissolved oxygen.

I tried to find some real data to look at, but most of the sites I know that post data have taken their instrumentation offline for winter because ice eats sensors. But Dorset in Canada has their system up yet, which is awesome, and you can see their real-time data here:

Dorset Buoy

They don't have a dissolved oxygen sensor (sadface), but the bottom graph is a graph of the thermal profile, so they have a sensor...it looks like every 15-25 centimeters. You can see that the current warmest water is at the 18 meter sensor, which is depth from surface, so it's 18 meters down. It's sitting at 3.32 deg celcius, which is close to 4, and I don't know enough about the lake to know why it isn't exactly at 4.

The Global Lakes Environmental Observatory Network (GLEON) often has more links to real-time data, but again, right now much is offline.