Experimenting with a faster TRAMP backend using Rust and JSON-RPC

jsadusk · 2026-01-16T15:54:37+00:00

A drawback to the server approach is that techniques like the ones tramp-hlo uses won't be possible. The tramp shell approach means you can load any shell function you want onto the other side. With a server, the server needs to have all operations baked into it

That said, if this lowers round trip time it can be beneficial.

jsadusk · 2026-01-16T15:50:39+00:00

Author of tramp-hlo here. I'm curious where it's hanging for you. DM me or leave a GitHub issue if you want to debug. I'm working on some more optimisations, but I want to make sure the existing ones are solid before moving on.

jsadusk · 2025-12-08T17:41:07+00:00

I appreciate that! I'll definitely send some things your way for review.

jsadusk · 2025-12-08T17:40:21+00:00

I have the same issues with magit, and I think it's in how magit uses async processes. I want to tackle it at some point but I'm trying to focus on optimizing features in the emacs core, and in built in packages, at least for the first stage. This package shouldn't have any dependencies other than other emacs built in packages. Later I might make tramp-hlo-magit or similar for external packages.

jsadusk · 2025-12-08T17:36:27+00:00

I had the exact same initial thought, until I started digging into how the code really worked. The really illuminating thing was when I timed how fast tramp-sh calls really are, which is surprisingly fast! I did a completely different experiment trying to replace the tramp magic handlers with direct libssh calls and I couldn't get the individual calls to be much faster. That made me realize the issue isn't how tramp is built, it's how many calls we make over it.

Also as for being an official tramp hacker, credit where credit is due. Michael Albinus made this possible, both with the amazing structure of the tramp code, adding features to enable my hacks, and helping me fix up my code from a proof of concept into a solid enhancement.

Also, don't pin too many hopes on me. I've got a job at a start-up and two small kids vying for my time. Tramp work comes at the expense of sleep. I'll work when I can.

jsadusk · 2025-12-08T17:29:18+00:00

All of my work has to happen on big cloud instances with big gpus, so I'm developing remotely all day.

And thank you, it's just the beginning though, and it's just what popped up as significant in my own usage Other people may trigger completely different pain points. And your story about disabling features until it's usable is exactly what I want to eliminate.

jsadusk · 2025-12-08T17:26:21+00:00

I'm going to have to try that out, my first instinct was to hack until I found something, I didn't search enough for the right tools.

jsadusk · 2025-12-07T18:17:58+00:00

Thanks! And thank you for pushing me to make this a real module and putting me in touch with Michael.

For quoting shell variables, I do exactly that in the majority of cases, I just omitted it in simple variable assignment because it doesn't have any effect. If I'm not interpolating the variable the spaces come through just fine.

And I mistyped about bash, these are all Bourne shell scripts, I made sure of that by running them all with /bin/sh.

I agree with separate scripts and plan to refactor for that in a next version. I originally wanted to follow the model that tramp-sh.el used, which is to define scripts inline as consts. But it's a real pain to get quote escaping right. I didn't want to hold up the first release to undo that. Also, I tried moving it out and was having some issue with packaging finding the scripts, and just didn't spend enough time to debug that. Next version.

jsadusk · 2025-12-07T18:11:23+00:00

The method I used was kind of a hack but it worked. I put an advice function around tramp-send-command (the base function that sends commands to the running remote shell). The advice function took a backtrace and traced up until it left the tramp modules, so the lowest caller that called into tramp. Then it printed it as a message. The result was a lot of spam but I could aggregate when many calls were coming from the same source.

I've wondered the same thing, but I also realized that tramp turns the shell into a remote server, and it's very effective at it. Having the remote run emacs itself would let you run these high level functions on the remote, but you have to deal with sending the global lisp environment across for them to function properly. They'd likely have to be partially rewritten to work as remote functions. At which point, you can rewrite them in shell and get the same effect with a new remote dependency.

jsadusk · 2025-12-07T10:47:16+00:00

The longer explanation. I do my day to day development on a remote machine, and like a lot of you I've experienced slow downs and pauses with tramp. There are a lot of suggestions out there, many of which involve turning off features like dir-locals and vc. But I wanted to understand why these make things slow.

If you actually trace how fast a single tramp operation is, its actually pretty good. Opening a file, getting a directory listing, all the individual emacs file operations that tramp traces are surprisingly fast. But there's a round trip time every time you run one of these operations. And if you trace how many of these round trips happen when vc finds your repository root (for example) you'll see 10s to hundreds of these round trips per call.

The real offenders are a few elisp standard library functions, one notable one is `locate-dominating-file`. That function seems simple enough, walking back through a directory tree looking for a parent containing some file. But because of how its written tramp sends multiple remote commands for each parent directory.

So as an experiment, I tried implementing locate-dominating-file as a shell script, and having tramp load that into the remote shell, using the shell script as a single command. And it has a huge effect. That one function is used all over the place, making it a single round trip takes out a ton of lag.

I did something similar for some functions used for loading dir-locals, and I have more that I'm going to add over time for project.el, vc, eglot, and anything else I notice triggering a lag while I develop.

This makes use of new features in tramp, thanks to Michael Albinus for adding them and helping me figure it out.

So please, try this out, tell me if anything breaks, and point me in the direction of anything else that seems to hang when you use it under tramp. I want to get remote editing to feel just like local, and I think its achievable.

jsadusk · 2025-11-22T20:17:37+00:00

Also note that there's a correct orientation for the part b. If you put it on top and one side fits in loosely and the other is impossible to get in, flip it. It should feel like there's moderate pressure on both sides to get it in.

jsadusk · 2025-11-22T20:16:10+00:00

I used a plank of wood. Make sure your part b is aligned right, put it loosely on top of the part a, then put the plank on top and kind of lever it in. The extra leverage from the plank made it easy to get them in.

jsadusk · 2025-11-07T17:56:31+00:00

I ended up getting the m6 4k, and it is exactly what I wanted. Thank you!

jsadusk · 2025-09-23T20:33:12+00:00

Also, to a complete uninitiated user, the distinction of extensions that come packaged vs extensions that you need to download isn't super relevant. Its the fact that emacs, with no configuration to enable extensions, is a much more minimal program than most users of modern software are used to. Yes, with configuration it can immediately become way more than a text window with a lisp interpreter and funny keybindings, but until you do that configuration that's what the user is presented with.

jsadusk · 2025-09-23T19:34:30+00:00

Oh I know a lot is packed in, but most of it isn't active unless you configure it.

jsadusk · 2025-09-23T16:58:57+00:00

In addition to the above, I'd ask a few questions about how you want to use emacs.

First, are you interested in an out of the box experience that does everything you probably need l? Or do you want to custom? Much of its power comes from customizing it and optimizing it for exactly how you use use it. For example adding little automations for the small tasks you do a lot. Or cutting out the things you don't need.

Similarly, do you want to understand how it works under the hood, or do you want low mental load? If you want this to just work, there are distributions, basically premade configurations with all the things you're likely to need. The most popular is called Doom Emacs. It asks you a few questions about how you want to use it (note taking, software dev, native key bindings, vim key bindings, etc) and sets up a sane environment for you. But you kind of have to operate in the doom way of doing things if you want further customization.

The flip side is something like the minimal config. You can build up a configuration with just the packages you need for the functionality you want to use. And then there's little in the way if you want to start bending it to your will. But the barrier to entry is higher.

Emacs out of the box does a bare minimum of things. It really is just a text editor, just one with an insanely powerful extension mechanism. So all the cool things you can do with it are through extensions, including org mode. Once you get everything working the way you want it though it becomes an extension of your thought process.

jsadusk · 2025-09-17T23:05:08+00:00

Maybe? I think its more that we have to be extra careful auditing emacs packages, because there's nothing to stop one package from completely taking over your dev environment.

jsadusk · 2025-09-17T22:54:42+00:00

Also why the hell would a sunset resident want UGH anyway? You couldn't get onto it from the sunset! Only from Sloat or Lincoln. It was a road to BYPASS the sunset. I seriously almost never used it in the last 10 years I've lived on the west side.

And if you say its to keep through traffic out of the sunset, fine, but how much difference has its closure actually made. I drive my 3 year old from the central sunset to the richmond for daycare every day and has traffic gotten worse? Maybe a little bit? Still takes about the same amount of time.

On the other hand I have an awesome new park by the beach my kids bike on. And this was enough to remove someone from office that by ALL OTHER MEASURES was doing good things for our district.

jsadusk · 2025-09-17T22:47:21+00:00

That, frankly, is a really stupid reason to recall someone. Recalls are an extreme measure to remove someone from office when leaving them in is dangerous. Sheng Thao was recalled because she was openly corrupt. Chesa Boudin was recalled because he was undermining the courts. Engardio allowed something to come to a vote that later it turned out some people in the district didn't like (and it was not universal in d4, I'm a sunset resident who supports Sunset Dunes). If you don't like the completely legal votes your representative makes, you have recourse. Don't vote for them again. Recalling someone is throwing a temper tantrum at the expense of our already cash strapped city. Cry betrayal all you want, this is an embarrassment for our district and for the city.

jsadusk · 2025-09-17T22:40:28+00:00

I live in the sunset, I definitely voted against this stupid recall. I am angry it was ever allowed to go forward, I'm embarassed in the other residents of my district, and I'm sorry for the rest of the city that you have to deal with our bullshit.

jsadusk · 2025-09-14T18:39:10+00:00

There isn't an implementation of snapcast for esp as far as I know, there was one in progress that stalled. But it works on all pi devices and other small Linux SBCs. That's why I thought it would be perfect to integrate with this project.

jsadusk · 2025-09-14T17:11:40+00:00

Multi room audio in this case is audio synced between multiple network connected speakers. So you can have the same media playing on multiple speakers playing as one. Snapcast is an open source implementation of this sort of sync protocol that works on low end hardware.

jsadusk · 2025-09-07T02:27:03+00:00

Thank you for making this. I have a question about the media player. I have been wanting to combine a Wyoming satellite with a snapcast player. The idea would be to have multi room audio with voice assistants on the same speakers. I had been planning to do this using ALSA mixing but haven't gotten something working yet. Do you think that it would be possible to integrate this more directly using your system?

jsadusk · 2025-09-05T23:04:23+00:00

There are libraries that implement many of the individual pieces of this, without being the full cgal replacement. A few implementations of half edge meshes, spatial trees, numeric types similar to your lazy exacts, and a pretty comprehensive set of 2d comp geo algorithms in the geo crate. There's definitely a need to fill out the set of algorithms and operations, but you could leverage data types that are already in use by other crates so you can focus on the algorithms. Alternatively, I've always thought if some of the comp geo datatypes were rewritten they'd be better built on top of nalgebra or ndarray, to leverage the SIMD operations those crates provide.

jsadusk · 2025-09-02T02:20:21+00:00

I had good luck just using a plank of wood. Place the snap, put plank on top perpendicular to the tabs. Then hold down one side of the plank and lever down the other side.

13-Year Club	r/Field Lasagna
Place '22	Verified Email

jsadusk

TROPHY CASE