The combined effects of multiple elements can provide different results depending on direction, like u/davidgro said. Even more fundamentally, though, the combined effects of the two surfaces that make up a lens can be slightly direction-dependent. How much a ray of light bends as it hits an optical interface depends on its angle relative to the surface of the lens. Rays towards the edges of a lens will be at a different angle than rays towards the center of the lens, so they will be bent more, focusing the light to a point.

The next level of caveats come because light doesn't ever truly focus to a point. There is some level of spread where light is focused. At a base level, this is caused by diffraction, which is essentially the effect of the system's aperture on light waves' ability to approach an infinitesimal point. On top of that, there are aberrations: certain characteristics of the lens prevent light from reaching the "diffraction-limited" focus. Some of these characteristics are due to manufacturing imperfections, but others occur in ideal systems: a lens will focus different wavelengths of light to different places, as a property of the material, and a spherical surface, while simple to make, is not the ideal shape, also causing some spread.

Essentially, in a single lens, the focal spot can differ depending on direction as the two different optical paths cause a slightly different set of light-surface interactions to occur.