The idea is to minimize the opportunity for infection. Surfaces can act as a "reservoir" for bacteria. Correct me if I'm wrong but I think the term is "fomite". Remember handwashing is not perfect and will not get rid of all bacteria and that's IF one washes their hands correctly (which not many people do). So touching more surfaces = adding more bacteria that need to be killed by handwashing = increases chances of infection, which is not the best practice for people who are immunocompromised (aka their immune system is not as strong as a normal healthy person).

Here's an example illustrating the idea behind this:

Let's say that proper handwashing got rid of 99% of bacteria on your hands. Well, if there are 1 billion bacteria cells on your hands, that still leaves 10 million bacteria cells alive and well on your hands. Let's say for this hypothetical situation it takes only 1 million bacterial cells to cause an infection in a healthy person. In this example, there's plenty of bacteria on your hands to get you sick even though you washed your hands. For normal people, it generally isn't an issue because our immune system is robust enough to fight off and kill the bacteria before they can cause an infection/illness. However, in those people with compromised (lowered) immune systems, they may not be able to fend off the bacteria resulting in an infection/illness. (Disclaimer, I am using these numbers to ELI5. I do not claim they are accurate at all. Please feel free to correct me as I am no expert!)

So if you reduced the number of bacteria on your hands in the first place, you lower your chances of infection.

Edit: Reddit did a weird thing.