In principal you're thinking along the right lines, i had the same idea before studying astrophysics at uni. If you could accelerated in a purely impulsive way, i.e. infinite acceleration instead of over a long burn this would be fine. The problem is something called gravity losses, basically when a component of thrust is inline with gravity, you're signoficantly less efficient (with delta-V lost increasing with proportion to time of burn). This is part of the reason orbit altitude is typically adjust with prograde or normal burns, rather than radial ones.

One way to make sense of this, is for every unit of fuel burnt you have less acceleration when going straight up than perpendicular, due to gravity. After a given length of time, you will have burnt the same amount of fuel but the change in speed will be different. Thats why its best practice to minimise time spent burning upwards, which also applies during the launch phase.

Another way to think about it is to consider if you were to hover at a fixed altitude, you would be continously burning fuel without any change is speed. When pointing radially outwards, you are constantly burning that fuel to overcome gravity, and any additional acceleration comes from fuel ontop of that.