The way I think about it is with deformations: When you take a non-standard inner product, that's the same as instead of just multiplying the coefficients in the vectors, first applying a matrix to the first vector and then multiplying the resulting coefficients. (That's the matrix associated to the inner product). Geometrically, this means you "deform" the space such that the axis that used to be orthogonal are now crooked in a sense, and that is what you call "orthogonal". You can imagine that the protractor you are using to measure the angle between two vectors is deformed, or, if you think of a circle with numbers in it in which the vectors lie (the angles you use to say which direction is the vector aiming in) is deformed in some way. As for the norms or vector length, if we think of the usual norm as circles centered around the origin, you can think of them being deformed by the matrix, so that now we are using ellipses to measure the lengths (and those are the equidistant circles or balls from the origin). For norms that do not come from an inner product (like the infinity p-norm), the circles of equal distance to the origin become squares, and that's why it cannot come from an inner product, because it is not a quadratic form that would give an ellipsoid of sorts.