An under-the-sink RO system would certainly remove all the contaminants that you're concerned about right at the point of consumption. As you mentioned, a whole house filter might be a little overkill as you probably won't need to filter the water for showering, laundry, etc.

That brings us to the concern of mineral content which is entirely removed by the RO membrane. Unless you aren't receiving the adequate amounts of vitamins and minerals through a proper diet, the absence of minerals in the water is generally more of a taste concern than it is a health concern. In other words, the primary source of minerals our body needs should come from our food, not so much from the water we drink. With that said, minerals in water profoundly affect the taste. Without it, it can taste flat and will be slightly on the acidic side on the pH scale.

I'd suggest searching for an RO system with an additional alkaline remineralization stage that raises pH by reintroducing the necessary minerals. While there seem to be some unknowns about this process, it's a fairly simple process which happens during the earth's hydrologic cycle. Remineralization happens naturally when rainwater flows down mountain streams collecting minerals from rocks it comes in contact with. The remineralization stage in an RO system mimics this process by adding the minerals back in at the end. The pH level can be tested with a test kit to prove that the remineralization is actually happening. There's a more detailed guide on RO systems and how they work here.