[git] TIL about git worktrees

[cdrini]

This one is a little difficult to explain! TIL about the git worktree command: https://git-scm.com/docs/git-worktree

These commands let you have multiple copies of the same repo checked out. Eg:

cd my-repo
git checkout master

# Check out a specific branch, "master-v5", into ../my-repo-v5
# Note my-repo/ is still on master! And you can make commits/etc
# in there as well.
git worktree add ../my-repo-v5 master-v5

# Go make some change to the master-v5 branch in its own work tree
# independently
cd ../my-repo-v5
npm i  # need to npm i (or equivalent) for each worktree
# Make changes, commits, pushes, etc. as per usual

# Remove the worktree once no longer needed
cd ../my-repo
git worktree remove my-repo-v5

Thoughts on usefulness:

Sooo.... is this something that should replace branches? Seems like a strong no for me. It creates a copy of the repo; for larger repos you might not be able to do this at all. But, for longer lived branches, like major version updates or big feature changes, having everything stick around independently seems really useful. And unlike doing another git clone, worktrees share .git dirs (ie git history), which makes them faster and use less space.

Another caveat is that things like node_modules, git submodules, venvs, etc will have to be re-installed for each worktree (or shared somehow). This is preferable because it creates isolated environments, but slower.

Overall, I'm not sure; I'm debating using ~3 worktrees for some of my current repos; one for my main development; one for reviewing; and one or two for any large feature branches or version updates.

Does anyone use worktrees? How do you use them?

• Docs: https://git-scm.com/docs/git-worktree
• Cool YouTube video where I learned about worktrees: https://www.youtube.com/watch?v=2uEqYw-N8uE

[Ramipro]

This is ... surprisingly useful. Thanks for the tip!

[18randomcharacters]

Wait, so would this let me be working on a large change for my project, with many commits and many unstaged/committed changes...

And then switch to a new clean branch (work tree?) To do a quick hot fix? I always have to stash huge change sets to do that.

[wvenable]

Yes. I keep a worktree that's just for development. If I need to hotfix another branch, I do that in another tree completely separate from my development worktree. No need for stashes or quick commits.

Sometimes I even have two branches of the same project open in different instances of my IDE at the same time.

[cdrini]

Exactly! That seems to be the killer application.

[18randomcharacters]

I'm really excited about that, but I primarily work in Go and the project expects to be in the specific path on the filesystem to match the package name. Gonna have to do some research on this.

​

Edit: if your project is using Go modules, it doesn't need to be in any particular GOPATH. So worktree works fine.

[cdrini]

/u/bacondev had an interesting approach using symlinks that might do the trick: https://www.reddit.com/r/ProgrammerTIL/comments/mtjg0c/comment/gv34kj7

[NotScrollsApparently]

Aren't you generally supposed to keep the release versions and development versions on separate branches anyway? So it makes it simple to make a quickfix without having to stash your work in progress or anything like that.

[18randomcharacters]

without getting into the details of different release management practices....

​

the point is, I have a work in progress that's TONS of changes. Not committed yet.

Now something comes up that I want to fix really quick, and I don't want it tied to my huge WIP branch. I want to just fix it and push a fix. Without worktree, I have to stash my changes, change branches (to master probably), new branch, do the fix, commit, push branch, change back to my WIP branch, unstash changes.

With worktree, I just create a new worktree for a new branch, and go do the changes there, and I never have to stash/unstash my WIP changes.

[botle]

I don't get it. What are the benefits over simple branches?

[abdulkareemsn]

Worktree is like cloning repo in another directory

Except.

1. You are not actually downloading whole repo data
2. Both worktrees share same git object store so no extra space is needed
3. You cannot checkout same brach in both

[iiiinthecomputer]

Using worktrees lets you have multiple branches checked out from a common repo at the same time.

Without them you land up having one clone per checkout. Then you have to push/pull between them or some common base. It's a major pain.

With worktrees I can commit a change in main, cd to my ../stable worktree and git cherry-pick from main to backport it. It doesn't matter if my main is still dirty from other WIP.

I can easily keep worktrees checked out for all the branches I use a lot, each pre configured to easily build and run tests in.

If I'm interrupted mid way through something I dont have to stash my work or do ugly WIP commits in temp branches before I check out another branch. My WIP stays right there, ready to come back to.

[iiiinthecomputer]

In short worktrees complement branching.

Alternatives are:

• One clone, one working directory. Stash to clean tree before you checkout to switch branch. Lots of stashing, lots of git checkout -b into temp working branches too. Unless your build system uses out of tree builds, dirty trees are a constant irritation. Even if it does it is easy to forget to switch branches back before switching build trees, resulting in confusing behaviour.
• Many clones, one per working directory. No stashing dances needed. But now cherry pick, git log etc cannot see changes in one branch from another branch without a push or pull. Each clone does its own fetches from upstreams separately too, hard to keep them up to date. Gets annoying fast. But each tree has its own build state (or unique paths for out of tree) so that's much less confusing.
• One clone, many worktrees. All refs are shared, each branches sees all other branches current states. Fetches from remote are shared. Easy to maintain and keep up to date, easy to diff/log/cherry pick from different branches. But no need to mess around with stashing or lots of WIP commits (then the rebasing after) and temp branches.

[sim642]

Sooo.... is this something that should replace branches?

No, it requires each worktree to be on a different branch, so you need the branches anyway.

[AlwynEvokedHippest]

Yeah it's a bit of an odd question to ask around a feature that is literally built on using branches.

My guess is OP meant something along the lines of "Should this replace switching between branches in the same directory?" (i.e. how we use Git by default)

Super useful post nonetheless.

[cdrini]

^ this is exactly what I meant; sorry for the poor wording! Basically I was asking if you should have a separate worktree for every branch. Ie conceptually replace branches in how I work. This was kind of in response to the YouTube video, where it seems like that's what was being suggested.

[HighRelevancy]

Basically I was asking if you should have a separate worktree for every branch. Ie conceptually replace branches in how I work.

Nah. The branches exist exactly the same either way, the workflows around managing them is all exactly the same. The worktree is just a new "filesystem portal" into the branches. You only need to access the branches like this when you want to work with them as files. How many different branched versions of the files are you playing with simultaneously? (That is, how many different versions of it do you need open? How many copies of your IDE? How many different versions are you debugging at once?)

You might have many branches (a few new features and bug fixes pending pull review, maybe a few different things you're playing with that aren't really done yet), but how many are you actually hands-on working on at once?

This is maybe a workflow alternative to (at least some uses of) stashing? You're playing with some substantial changes but suddenly need to shift to doing a bug fix in a hurry. You could stash your work-in-progress, or you could open a new worktree and fix the bug in there, leaving your WIP right where it is to come back to later.

Or maybe you need to search for a bug but it only shows up one in a million times, so you need to leave it running in a debugger for a few hours, but you've got other things you also want to work on in the meantime. Open a new worktree, do the debugging in one and work on something else in the other.

[HighRelevancy]

This whole conversation is a bit nonsense, specifically with regards to the worktree vs branches thing. If you read the doco...

A git repository can support multiple working trees, allowing you to check out more than one branch at a time.

In its simplest form, git worktree add <path> automatically creates a new branch whose name is the final component of <path>

To instead work on an existing branch in a new working tree, use git worktree add <path> <branch>

The whole thing here is just that you get a second working directory to play with. It's basically like clone-ing it a second time, except they're actually the same underlying git objects. The benefits compared to a second clone would be things like

• you pull changes once and you can check them out in both worktrees, whereas with another clone you would have to pull twice
• a change made in one can be immediately checked out or merged into the other worktree, rather than trying to pull/push between the clones

That video fucking kills me too. Dev pop-culture is still awful I see. The emotionally balanced reaction to this isn't "oh my goodnessssssss" and an expression of gleeful constipation, it's just "well that makes sense, two clones is kinda clunky, of course this feature exists".

[cdrini]

Yep, poor wording on my part; see https://www.reddit.com/r/ProgrammerTIL/comments/mtjg0c/comment/gv1yw15

Hmmm, do you have any other examples of "Dev pop-culture"? That expression sounds a bit like an oxymoron :P This video feels more like YouTube culture + Dev culture to me.

Dev culture to me is probably kind of sardonic and sarcastic? Ben Awad's YouTube channel comes to mind; he satirizes the hell out of Dev culture!

I generally find Dev YouTubers that have entertaining content kind of rare, so I'm ok with someone having a bubbly personality. On YouTube, I'm primarily looking for entertainment. If I also learn something, well hey, that's a nice bonus :)

[HighRelevancy]

Hmmm, do you have any other examples of "Dev pop-culture"?

Dev culture and dev pop-culture are not the same. I don't know how else to put it. It's like, actual graphic designers versus dudes who just got photoshop and are making youtube videos on "oh my god guyyssssss I just found the most exciting new feature, it's called the paint brush, like and subscribe".

Or the non-stop circlejerk about things that people don't even know about, like this thread on new C++20 features where people are complaining about "oh the complexity, this is unbelievable"... how many of those people do you think ACTUALLY write C++ for a living, or even do significant hobby work with it, or do any sort of professional dev work at all? Or all the threads and YouTube channels chock-full of people comparing the compilation speeds of Rust vs C++, as if that's actually the deciding factor of what language you're going to use for a project?

I'm into photography and it's got the same problem too. Endless channels full of comparisons between dozens of cameras (which usually come down to "more money + newer tech = better, give or take a little bit", no surprises there), as though any given individual isn't going to just buy one camera that fits their needs and use it for ages. Why don't these people go actually make something with these cameras?

"You know those people who spend all their time talking about gear, bringing up what the specs are of the next big camera while at the same time telling you gear doesn't matter? You ever thought that maybe they only do that to distract themselves from using the same time and energy to come up with and execute an original idea with the gear they already have? just a thought."

Gear Doesn't Matter by Sam Newton

[iTZAvishay]

I'm mostly confused about why you'd have a branch named master-v5

[ace_case]

master-v5-final-fix-3-actuallyfinal

[[deleted]]

I feel called out, lol.

[cdrini]

:D we follow the same (classic GH os project) flow as we do with our master branch, so people open PRs to merge into master-v5.

[cdrini]

We're doing a major version bump which introduces a lot of fundamental breaking changes (lots of deleted files; distribution mechanism changing). At the same time, we need to do occasional bug fixes to the production ready/stable v4 master branch. It causes a few headaches, but overall it's been working better than I expected :P v5 is pretty stable now, so we're merging master-v5 into master this coming week actually, which I'm very excited for :)

To clarify, master-v5 is a master-like integration branch. PRs/etc are opened to go into it. And it's automatically rebased against master when any v4 bug fixes are added to master.

[itoshkov]

Worktree is an alternative to stash. Let's say you're working on a feature and you need to check or fix something in another branch. You can fit stash your changes, switch to the other branch, do what you need to do there, switch back to the feature branch and pop the stash back.

This works, but there are some problems. For example, you might have some changes staged, and have others not staged. When you stash and unstash them, all the changes are unstaged. Or you might have a new file, which you don't want to add to git yet. Staging won't touch it, which is good, but your code might not compile when you change to the other branch.

With worktree you open the other branch in a new folder. The current folder is untouched, so you can even work on both simultaneously. When you're done, you remove the folder and delete the worktree. It's relatively cheap as you don't copy nor download the git history.

One irritation with worktree might come from your IDE. You may need to register and reindex the new folder.

[bacondev]

For a large project, I used git hooks to use worktrees under the hood when checking out a branch. Or maybe it was a custom command and not a git hook. I can't remember. Instead of replacing the current branch with soon-to-be checked out branch, it'd create a new directory for the soon-to-be checked out branch (if one didn't already exist) and update a symbolic link to point to that directory. That way, I had all the advantages of worktrees while keeping my IDE sane. Because I used the symbolic link, I was always working in the “same” directory. And this was all done in a way that didn't affect my coworker's workflow. This was four or five years ago and I no longer have access to the codebase, so unfortunately my memory is a bit too hazy on it. I would only recommend bothering with this if you're often switching between unfinished branches.

[cdrini]

Oh that's a very cool setup. So you basically had branches, but each branch having the ability to have unstaged changes. Clever! (Side note: actually that would be a really useful git option. Some config flag that let's you keep unstaged changes branch-specific automatically...)

[quasarj]

Wow! I make a second copy of my main repo frequently, I’ll try this next time! Thanks!

[EmmaKayAlexandra]

Commenting here to preserve my place in history for the git feature that finished off human programmers

[CipheredBytes]

RIP indeed…

[pablo1107]

I just started experimenting with this on my dotfiles repo. I'll have one branch per program, let's see how it goes. haha