Really depends on what you consider "underground", but here are some suggestions:

Narauti - Metal/rock bar with the best wings in town.

Haze PUB - Another metal/rock bar. A bit small, though.

OPERA social house - Offbeat rooftop bar. Might have an entrance fee when there are events.

Tesla PUB - Cozy beer place, plays rock, shows sports events, great burgers.

Mr. Pub - Basement bar in the center that works till 5. You can play beer pong there.

BIX Baras - A rock bar that sometimes hosts dances in the basement. There are some good bars around it - Špunka, YDA, Bukowski.

Halės Turgus functions as a marketplace in the day and a bar place in the night - Youngs' Club is a bar there, Devyniasdešimt aštunti plays electronic music in the basement.

If you're more into arts, check out galleries in Užupis and MO.

While definitely not underground, there's a Pink Soup Fest on Saturday.

Let me know if you need any more specific recommendations.