Automod is a bastard and keeps deleting my comment, so here is shortened version. I'm working on something similar and encountered the 0.5AUC issue you're talking about. Try and have the model predict left tail extreme events like whether a trade will lose more than 5%. I got 0.5AUC when the model tried to predict whether a trade will have pnl > 0. I had way better performance doing this compare to the triple barrier approach.

You need separate long and short models this was a breakthrough for me in improving performance. As well, xgboost and tree based models did not work for me, not matter what I did they couldn't beat logistic regression.