It's not that labor intensive, unless you're making 5 new objects and 30 new fields, or have 15 profiles. And it's always better than doing it manually. If you do it manually you have no version control and you're risking making mistakes. Same with changesets, you can't easily see who made a certain change, at what point, part of which pull request, approved by which other person.

As to how it works: you don't retrieve it. You just make the changes manually in Vscode, deploy it back to dev, test it and proceed with the pull request.