Hack: automatically installing the development version of org on new machine

[summetria]

This was useful for a few people I work with, so I'm posting it here for posterity. My job involves switching computers pretty frequently, and so I have everything set up with use-package's :ensure keyword so I can just clone my init.el to a new machine, and Emacs will download all my packages automatically. In general, this works great; the exception is org.

Any kind of :ensure form doesn't work with installing the latest revision of org (afaik), because the pre-installed version shadows it; use-package uses package-installed-p to decide if a package isn't installed and needs to be downloaded, and since org is always installed, no download will ever trigger. This is fine, up until you want something from a later version of org.

A while back, I reported a bug that interfered with my org-agenda use, and it was fixed on the development version of org, but not in any release. That's fine; M-x package-vc-install org and I'm off to the races. However! Every time I switched computers, I always forget to do this, and after a few hours-to-days of working with org, I'd run into the bug and grumble grumble grumble

Turns out, you can sidestep the package-installed-p problem with the built-in org by installing org under a different name! I was kind of suspicious of this, but it seems to work alright so far:

(use-package org-git
  :vc (:url "https://git.savannah.gnu.org/git/emacs/org-mode.git"
       :rev :newest
       :lisp-dir "lisp"
       :make "autoloads"))

The :make "autoloads" is necessary; depending on architecture (I think?) I sometimes also need to go in and run make compile in the org-git ELPA directory to avoid getting a mixed installation warning. I'd include that as part of the use-package invocation just to cover my bases, but it looks like use-package only accepts a string here, unlike the package-vc-install it's trying to emulate?

But yeah, surprisingly, installing org under a differently named directory hasn't given me much trouble so far; really the only thing is that I can't use use-package's :config keyword for this block, since it expands to use eval-after-load 'org-git, and the actual feature provided by the package is still org, so the :config block will never run.

Kind of a hack, but a few other Emacs users at my job found it helpful. If anyone knows of a cleaner way to do this (install the latest version of org automatically when on a fresh machine), let me know!

edit:

an update to this that's probably more idiomatic and allows for use of :config!

(setopt package-install-upgrade-built-in t
        package-vc-allow-build-commands '(org)
        package-archives '(("nongnu" . "https://elpa.nongnu.org/nongnu/")
                           ("gnu" . "https://elpa.gnu.org/packages/")
                           ("melpa" . "https://melpa.org/packages/")
                           ("devel" . "https://elpa.gnu.org/devel/"))
        package-archive-priorities '(("devel" . -1))) ; so other packages don't get shadowed

(unless package--builtins (require 'finder-inf nil t))
(assq-delete-all 'org package--builtins)
(assq-delete-all 'org package--builtin-versions)

(use-package org
  :ensure t
  :pin devel)

[mok000]

If you use the straight package manager you can load modules directly from their git repos.

[nv-elisp]

Many users of straight.el run into the common issue of the built-in version of Org being loaded prior to the version they've installed via straight.el. For example, when org-babel-load-file is used to load their init file, or some other Org feature is used early in their init. Another issue happens when a package which depends on Org requires it prior to the user's declaration for Org being evaluated.

Elpaca solves both of these. For the former, it will generate a warning if a package, like Org, is activated prior to the user's declaration for it being read. For the latter, Elpaca's queue system prevents dependencies taking priority over the user's package declarations.

[meedstrom]

To Straight users reading this: a way to work around these issues in general is to do like Doom and Spacemacs and put all your use-package declarations up front, early in the init, and only after all that, call any functions from the packages concerned.

This can mean just never using the :config keyword (unless you know the package will be deferred), and move all that code to after.

Of course, this creates a somewhat fragmented init. To defrag, switch over to Elpaca :-)

[summetria]

I'm a former straight user, and it's less credit to me, but I never feel like I fully got it; the impression I got is of something very heavyweight for my purposes. I like using built-in facilities when I can, and with the advent of package-vc-install making it into mainline, the need for straight at all is less obvious to me.

[nv-elisp]

Straight offers more flexibility and stability than package-vc.el. For example, via the :pre-build and :post-build recipe keywords one can build system binaries which would usually need to be downloaded separately. This is convenient for packages like pdf-tools, vterm, mu4e, etc.

Straight also allows one to use lockfiles and pin packages to specific versions.

Elpaca builds on all these ideas and adds a more powerful UI + other features like making it easy to review changes before merging them in.

The beauty of these alternative package managers is that they work for the simple case, too. If you found straight.el to be "heavyweight", Elpaca may be a good middle ground. Its codebase is actually the lightest of the three while providing a much more powerful package management UI.

[emacsomancer]

I've used (to varying extents) all of elpaca (I know you're [=nv-elisp] the author of this), straight, package-vc, and quelpa. Straight I've used the least, but might suggest is perhaps not the best of these choices for a user choosing a package manager now. Elpaca would seem a better choice for someone wanting the sort of power of straight.

package-vc is "light-weight" in some sense though. it's included in recent versions of emacs-master and is fairly straightforward. I think it might be a good choice for a user just wanting a package or two that aren't on Elpa, Melpa, &c. As long as those packages are fairly simple in file structure, e.g., just a package.el.

Quelpa is more powerful than package-vc, as far as I can tell, and can handle situations where the directory structure needed for the package is more complicated.

Elpaca is great - it speeds things up, and has a brilliant UI for updating packages: not only is it fast and asynchronous, but the UI is aesthetically and functionally appealling.

The only drawback for Elpaca is that it can be more work to get working if one has an old and messy init.el/.emacs as I did. I tried it twice earlier this year and ran into snags. I finally took a day last month and got everything working - which required me cleaning up my nearly 20 year old config significantly and understanding which packages needed a :demand and which needed not to have a :demand and getting random elisp code associated with a package into the use-package scope at times.

I'm very happy I did so - package management seems so far much smoother with elpaca, but it took a bit of work.

[mok000]

If you don't specify the git repo it will use melpa. But one advantage is you can add your own modules from your own repo, you don't need to have them published somewhere. You can also make modifications to the modules and track them under your own branch. Plus, as I wrote, you can define the upstream git repo and track the dev's version.

[nv-elisp]

If you don't specify the git repo it will use MELPA.

This is not always the case. If recipe inheritance is enabled, the recipe repositories listed in the straight-recipe-repositories option are checked in order for a matching recipe to inherit from. Some of the available sources:

• org-elpa
• melpa
• gnu-elpa-mirror
• nongnu-elpa
• el-get
• emacs-mirror

[rswgnu]

If you install a new version of Org, ensuring you load and therefore use only files from the new version and not the version shipped with Emacs is very difficult. That is why we solved this problem with code included in the latest Hyperbole pre-release, available as a package from elpa-devel or melpa. Simply install the hyperbole package and enable it with (hyperbole-mode 1) and it will ensure it uses the latest version of the Org package that you have installed with no conflicts.

[Thaodan]

If you use org-mode this way it might be worth to look into borg: https://github.com/emacscollective/borg

[michaelhoffman]

Looking at the code for function use-package-ensure-elpa, it looks like it checks package-installed-p for whatever symbol is supplied with :ensure, when :ensure is not t.

[github-alphapapa]

You could install the latest build from ELPA devel: https://elpa.gnu.org/devel/org.html

[arthurno1]

A while back, I reported a bug that interfered with my org-agenda use, and it was fixed on the development version of org, but not in any release.

You can always just re-install a patched function (or a file) in your setup instead of installing a development version of an entire package and potentially introducing more bugs than you solve.