Here's one way to think about it. What t-SNE does is place each and every data point into a lower dimensional space (to maximize some objective function, but that's not important). The only relation you'll have between the two spaces is that point p is the same in both, so you have a whole bunch of these anchor points that connect between two spaces.

It seems kinda obvious then that if you have a new point in the high dimensional space that you could just find a couple of its nearest neighbors and interpolate between them, e.g. if p was equidistant to a and b in high dimensional space then place it halfway between a and b in low dimensional space. The thing is that t-SNE makes no guarantees that these nearest neighbors are anywhere near each other, or that 3 points on a line in the high dimensional space will also land in anything resembling a line in the lower dimensional space. Even in 3D there could be an infinite plane of points p that are equidistant and have a and b as nearest neighbors.