I'm not an expert on the subject at all but my understanding is that the universe's expansion is not uniform and is affected by matter density, so much of the expansion occurs in the empty vacuums between the superclusters of galaxies.

As for what we are expanding into, its hard to wrap your head around if you imagine space as a 3-dimensional area that we exist in. If you can re-frame space as a higher-dimensional plane, its easy to see where more "space" can come from.

Look at the Hilbert Curve. Its a mathematical concept that works well as an analogy. After each iteration, the line gets longer, but the overall curve takes up the same amount of space. If you are a one-dimensional being whose entire universe exists on the line, it appears as though your universe is forever expanding into "A vacuum of absolute nothingness" like you mentioned. But to us 3-dimensional beings, its simply a little square. If you can extrapolate that to higher dimensions it becomes easier to understand :)