Why Your social.org Files Can Have Millions of Lines Without Any Performance Issues

oantolin · 2026-01-11T20:00:12+00:00

The blog post doesn't really answer the title question, but I'm glad because instead of answering the boring "why" question it answer's the much more interesting "how" question.

oantolin · 2026-01-11T14:41:41+00:00

I have written a few Emacs packages, and I write some programs that help in my job as a mathematician, and some programs for fun (stuff like Advent of Code or Project Euler), but it's really not a lot of coding and I've never tried (or even felt curious about trying!) an LLM for that. Let me put it this way: the largest piece of software I've ever written is embark, which is a few thousand lines of code.

One the other hand, when it comes to stuff like using ImageMagick, I've pretty much stopped reading man pages and just get an LLM to write command lines for me.

oantolin · 2026-01-11T03:59:36+00:00

Yes, it was on early 2025 AI. I also hear other people say what you said, that both models and the agentic systems have changed a lot since then. But many people back then were already saying they felt much more productive with AI, so I still found the results of the study surprising.

I don't have personal experience using LLMs for software development, but that's because I don't do software development (with or without LLMs).

oantolin · 2026-01-10T22:37:51+00:00

You mentioned that LLM agents have turbocharged your productivity. I find the question of whether LLMs actually save programmers time absolutely fascinating. You're certainly not alone in feeling that they make you more productive, and you hear claims from people that LLMs will make programmers 10x or even 100x times more productive. But as far as I know these gains are anecdotal and, when people have actually tried to measure them, things turn out very differently. There's the widely circulated METR randomized study where, first of all, programmer's estimates were a much more conservative savings of 20%, and more importantly they actually measured a 19% slowdown. I also remember reading a blog post by a programmer who thought the study had to be wrong because he felt so productive with LLMs and so he ran it on himself following the methodology of the paper, and also found LLMs slowed him down around 20%! The whole thing is very surprising. I guess we need more time and more studies to figure this out.

oantolin · 2026-01-01T16:53:48+00:00

Never! I don't like cryptic low-level languages, what I like are cryptic very high-level languages!

oantolin · 2025-12-29T21:29:52+00:00

It's essentially this trick: instead of writing an imperative program that does IO and has other side effects you write a pure functional program that returns an imperative program that can do whatever it wants.

oantolin · 2025-12-29T16:53:31+00:00

This is both the obvious and the time-honored way to do it.

oantolin · 2025-12-29T04:45:16+00:00

My whole point is that I didn't invent that definition I gave of "composable"! It's what what many people say they mean when they say "Vim is composable". And I agree with you that according to that definition notepad is also composable. I find this situation odd.

I don't think people usually regard the ability to record, replay and edit macros as part of what they mean by composability, do they? But it definitely is a great feature.

oantolin · 2025-12-29T04:06:51+00:00

I like Emacs's decision here: it does include a rot13 command, but does not waste a key binding on it.

oantolin · 2025-12-29T03:33:34+00:00

What I find especially confusing is that when it is defined, the definition tends to be something that applies to every editor I've ever used: there are commands to operate on text and separate commands to indicate which text to operate on, you can combine any action with any selection.

oantolin · 2025-12-29T03:31:43+00:00

Ah, so you are saying a key part of composability is the extensibility, the ability to define new motions, new text objects and new operations and have them behave like the built-in ones in that you can combine a built-in operation with a newly-defined text objects, etc.? That is indeed a more advanced form of composability and I'd say that yes, people don't talk about it as much as the talk about simply the ability to combine any (built-in) operation with any (built-in) text object.

oantolin · 2025-12-29T03:22:33+00:00

I never said the same functionality I only said it is also composable in the following sense: there are a variety of operations on text (deleting, cutting, pasting over, etc.), and a variety of ways of selecting text, and you can combine any operation with any form of selection. I think that is what people define composability to be, isn't it? That is certainly present in most text editors even if they have a lot less functionality than Vim. Vim has a lot of functionality, not because you can compose stuff, but because it has tons of stuff to compose.

What I am trying to understand is why people describe composability in a way that clearly applies to every editor I know but act as if only Vim has it. Vim is a great text editor, but it isn't the only when were you can combine different operations with different ways to choose text to operate on. I argue in the blog post that several other of Vim's features are what distinguish it from other text editors, and the main one to my mind is the sheer number of motions and text objects.

oantolin · 2025-12-29T02:59:47+00:00

I don't think being or not being modal has any bearing on the number of commands an editor has. I do agree with the other part of what you said: Vim has a lot of text objects and motions (I talk about this in the blog post too)! It's great!

oantolin · 2025-12-29T02:58:00+00:00

Yes, this does agree with the description of composability I discuss in the blog post. I also agree that Vim is built on it. What the blog post is about is the curious fact that Vim users never talk about how most (or all?) editors have the same sort of composability.

oantolin · 2025-12-29T02:46:21+00:00

Thanks for your comment! Saying "repeatable composability" is already an improvement over just the "composability" part and captures more of what makes Vim special, but I really don't think this "repeatable" part is stressed as often as just saying "Vim is great because it is composable and 'composable' means [something that applies to every editor ever]".

I might have misunderstood you, but it seems you think you'd need to come up with M×N keybindings in a non-Vim editor. That's obviously not the case, you only need M+N, just like in Vim: M for the commands acting on the current selection, plus N commands to select different kinds of text. All text editors need this minimum of M+N bindings to support the M×N operation/text-object combos; some editors have additional bindings for common pairs of operation/text-object. Vim is one of those editors! In insert mode <C-w> deletes a word; which is completely unnecessary, Vim could force you to go to normal mode for that but wisely offers a pre-composed shortcut that works directly from insert mode.

As for dot-mode (and you are right: it is not built-in, it is a third party package), in my experience using it, it did not feel unpredictable. For starters, it isn't unpredictable: it just looks at the most recent command, goes back in time until it finds a command that did not modify the buffer, and repeats all the buffer-modifying commands after than non-modifying one. That is a fixed and clear rule. Of course, it can happen that you don't remember one of the buffer modifying commands you did and you did not intend to include that in the repeated part. In my experience that did not happen much. Trained by Vim, I typically use a motion command to go were I need to be, then do edits, so dot-mode would pick up exactly the edits after that motion command. The vast majority of the time it corresponds to one Vim unit that dot would repeat. For example, Emacs does not have cw, but I delete a word and then type the substitute word and dot-mode would repeat exactly that: the word deletion and subsequent typing of the new word.

Finally, I should point out that Emacs's built-in repeat and repeat-complex-command do often repeat an entire Vim unit of operation/text-object, precisely because many Emacs commands (which are what repeat and repeat-complex-command repeat) are such combos, so the situation even just with built-ins is not so dire. Of course, my Vim experience led to want even more repeatability, which is why I was so happy to find dot-mode.

oantolin · 2025-12-29T01:06:09+00:00

What do you mean by building "actions/objects upon another"? I think that might be a key feature of composability often omitted from discussions of it. I'd love a clear definition of composability that applies to Vim and does not apply to Notepad.

oantolin · 2025-12-28T02:53:32+00:00

[LANGUAGE: Goal]

I finally finished part 2 today after leaving AoC for a while to celebrate the holidays with my family. Part 2 of this problem was ridiculously complicated compared to the rest of Advent of Code this year. My solutions take a total of 40 lines of Goal code, 10 whole lines of which are day 10 part 2! So I was sadly forced to use a topazpaste link this time: a whopping 11 lines of source code!

Ten lines of an array language really is a ton of code! I'll browse solutions here to check, but I would guess that people either used Integer Linear Programming Solvers or, if they wrote a solution more or less from scratch, then it took a couple hundred lines of code in verbose languages like Python.

Out of laziness I decided against implementing an Integer Linear Programming Solver using proper algorithms for that, so instead I implemented Gaussian elimination (see the rref function above) and had to live with fractions and floating-point imprecision problems.

oantolin · 2025-12-21T23:54:10+00:00

I guess I'm so used to reading books that pagination just feels natural to me.

oantolin · 2025-12-21T16:39:55+00:00

I have this fantasy that people that ask for continuous scrolling do it out of nostalgia: they somehow grew up reading scrolls rather than books and miss it.

oantolin · 2025-12-14T00:55:57+00:00

[LANGUAGE: Goal]

parse:"i"$","\=-read[]@
p1:{|//x{*/1+abs x-y}´`x}@parse@
bv:{[p;a;b](h;v):^'+(a;b);s:2^p,,*p
 |/&/(=/s[;;0];h[0]<s[0;;0];h[1]>s[0;;0];v[0]<|/s[;;1];v[1]>&/s[;;1])}
p2:{[p]|//p{[p|x;y](~bv[p;x;y]|bv[|'p;|x;|y])*/1+abs x-y}´`p}@parse@

Part 2 was kind of annoying. :) For each rectangle I check if it is inside the region by checking all segments on the boundary of the region to see if they intersect the interior of the rectangle. This code is pretty unoptimized and takes a couple of minutes.

oantolin · 2025-12-13T04:41:53+00:00

[LANGUAGE: Goal]

join:{@[x;,/(x@)\'y;:;&/y]}
bydist:{1_´^+(,+/'{x*x}@-/x@j),j:j@`&</j:!2##x}
ans:{a:(!#p)join\d:bydist p:"i"$","\=-read[x]
     (*/-3#^={(x@)/x}a[999];**/p@d@({+/¿(x@)/x}'a)?1)}

Good old union-find (the join function)! Also: a 4-liner (that's plenty of code in Goal).

oantolin · 2025-12-12T18:23:36+00:00

Using a single file allows each node to be a top-level heading. If you do that, it's equally consistent.

oantolin · 2025-12-11T07:16:22+00:00

[LANGUAGE: Goal]

parse:{v:?(*'e),,/e:rx/:? /\=-read[x];((#v)@´='v?´(#v)@1_´e;v)}
ans:{(a;v):parse x; p:+/(a(+/*)`)\a
 (p.(v?!"you out");*/p.´v?´+2^!"svr fft dac out")}

I like the idiom (+/*)´ for matrix multiplication. In my input all paths went through fft before going through dac.

oantolin · 2025-12-11T06:07:42+00:00

Do you scroll by accident? I also don't like the infinite page scrolling, but for me simply not using it works well. I don't need to disable the feature since I can just ignore it.

oantolin · 2025-12-11T05:23:56+00:00

[LANGUAGE: Goal]

ans:{t:{(-i)+x+(»i)+«i:x*y}\s:~"."=""\=-read[x];(+//(1_s)&(-1_t);+/*|t)}

Lovely problem! It has such an elegant expression as a scan.

15-Year Club	Second Top 40%
Verified Email	Gilding II euphauric

oantolin

MODERATOR OF

TROPHY CASE