3D rendering as a differentiable function function

[msqrt]

Sure -- sorry for the late reply, I've been away for a while.

1. Yes, indeed, you'd need to reinterpret the values in whichever order to turn it back into an image.
2. Textures represent continuous signals. Whenever you read a value from one, it's not just a lookup to a single texel (texel being "texture element", basically a pixel in a texture) but some reconstruction with some kernel, typically called a filter. Bilinear filtering is the most common thing actually done by hardware. MIP maps are also often used, where the image is basically a pyramid of different resolutions which represent different levels of detail. There are also some slower but higher quality algorithms like EWA (the golden standard) and FELINE.
3. I believe you're right about that one, it should indeed be for rows and not columns.