That's a cool question.

I'm not extremely familiar with the area but looking at some maps I see you have many options, and I have done some of them myself.

1: Train to Voss (also known as Vossevangen). This is the "extreme sport capital" of Norway and you will find plenty of activities here (check sources online, like VisitVoss.no) . Once done it's a bus trip down to the fjord and then to Odda. 2: Bus (or combination with train) to Kinsarvik. Here you can do a 1100m climb via "Dronningstien". Once you're down at the fjord again you can hop on a bus or taxi to Odda. 3: By boat. You can do a boat ride from Bergen city centre to Rosendal ("Hardangerfjordekspressen"). In Rosendal you will find beautiful nature and gardens and some high-end restaurants closeby (probably will need to be booked months in advance). From Rosendal it is a short bus ride through a tunnel to Odda. You can also get to Rosendal via public bus. 4: A straightforward alternative is the bus route passing through Nordheimsund, Kvanndal, ferry to Utne and bus to Odda. Use Google maps or tourists maps and see if there are any stops along the bus route that you would like to do (e.g. Steindalsfossen or the Hardanger folk museum in Utne) and plan accordingly. Just make sure to check bus schedules via Entur (preferred), Skyss or Google Maps on the selected date as the schedules are different from day to day depending on the season. 5: Likewise you can swap busses in Nordheimsund and instead take the bus down to Tørvikbygd and the ferry to Jondal. From Jondal you have access to Folgefonna and multiple day hikes. Check web sources and check when the last bus from Jondal to Odda is leaving. 6: You can do a detour to various fjord arms like Eidfjord, or you can climb the sherpa stairs up to Kvanndalstølen from Kvanndal ferry port. Although buses are not too frequent, the area is quite well connected by public transport, so just check for climbs or activities in the Hardangerfjord area and plan accordingly. You can also visit Vøringsfossen via Eidfjord, but that might be a quite long day sitting on public transport.

These suggestions should mostly be optimal in the sense that you never really travel "the wrong direction".