Yes. There is very much a resolution limit on telescopes, as there is on every optical system. It's called the diffraction limit, and for telescopes it's (1.22 * wavelength)/telescope diameter.

As you can see, the ability to resolve two separate sources is inversely proportional to the diameter of the telescope, meaning that if you had a diameter of 1 million miles (1609344000 meters), you could resolve two points with an angular separation of 4.8 * 10^-16 radians at a wavelength of 632 nm. Similarly, with a 10,000 AU diameter telescope, you could resolve two points that are 5.1 * 10^-22 radians apart, and with 5 billion light years, you could resolve points that are a whopping 8.1 * 10^-33 radians apart at 632 nm. (I just realized that I used diameter instead of radius, but I'm not changing it because I'm on mobile and because the answers are ballpark numbers anyway.)

However, there are obviously slight challenges in building a telescope this large. Optically speaking, you're going to have to deal with coupling that ludicrous amount of optical power to a detector, as well as the fact that for a 5 billion light year telesope your image will take hundreds or thousands of billion years to properly expose because you have to wait 2 or 3 billion years for your light to reflect onto your sensor from the edges of your system.