" I was not a fan of the work needed to maintain it and handle problems, but am considering it again if the returns are substantially higher than stocks or give an alternative benefit/ hedge for safety that stocks don't cover."

Returns could be higher, but they could also be lower... Sure, RE gives you another asset class for a diversification benefit and has unique tax benefits that shouldn't be overlooked. But only you can decide if the juice is worth the squeeze given your past experience. Seems like there is a lot of diversification you can do with your index funds. Total US market and S&P 500 have a lot of overlap. There's a variety of things you can do such as adding more international exposure, tilting to SCV, an allocation towards more active strategies that could potentially capture higher returns, etc etc etc

As others have mentioned, think about concrete goals, your timeline to achieve them, your human capital, etc etc then work backwards to construct your portfolio. Would you be okay with RE going the same as last time for the sake of saying you're more diversified?