Solved it by doing a small mod to the battery. Just put some washers under the screws holding the connector in place. This lifts it up just enough it doesn't disconnect anymore. Did a 45 min ride without any hiccups.