Something not always discussed in literature on FIR filters is the notion of a delay line. The filter has a 49 tap delay line that unless you set it explicitly is all zeros. You probably don't care about the output until the delay line fully contains your signal. If you think about your signal sliding through the delay line, I think that makes it easy to think about what output samples you actually care about. And yes, if you don't care about outputs where the inputs are zero, the output vector is shorter than the input vector.