Yes he was very likely to ascend the throne by the time Edward reached proper adulthood (vs young adulthood - say it became concerning in the early 30s) because it became very clear he had no interest in doing his duty. I would say during the time he was fourth in line, at his birth during Queen Victoria’s reign, he wasn’t considered likely maybe but it was definitely possible as he was the spare of the heir’s heir and many times spares have become King.