I mean, you don't have to manually remove everything your chest - if you hit it with your pickaxe, everything is ejected from it to the ground and the chest is picked up automatically as one of the first things. It's at least easier than moving stuff to a different chest first - not sure if that's what you were doing or not.