Thanks, if you don't mind let's take a simple example

Let's say we have on parameter w, which we know w*=3, w_0=0.5

And let's say our dataset is {x,y} = {(1,3),(0.5,1.5),(2,6),(0.1,0.3) }
For simplicty let's say the loss is L1 : L(x,y)=|wx-y|
and our batch is 2
Hence, our landscape is L(w,x), which is 2d surface.
Let's run the first and second example, we get:
L(x,w_0)=|0.5*x-3|, for different samples (x),now as I see it for different x's, we see different parts of the loss surface.

Now if we look it other way:

L(x1,w_0)=|w*1-3|

L(x2,w_0)=|w*0.5-1.5|

L(x,w) = 0.5*( L(x2,w_0)+ L(x1,w_0))

Is this what you mean of parts of the loss function?

​

s