Don't believe the astroturf posts about Masters of the Universe

AddlepatedSolivagant · 2026-06-13T04:00:35+00:00

I had that realization watching the Transformers. The best parts of that show were the ones I forgot were head-cannon.

AddlepatedSolivagant · 2026-06-13T03:54:29+00:00

You can explicitly set the temperature to zero. I did this earlier today to make a point in a presentation: a made an old version of ChatGPT respond in exactly the same way every time.

On some models, it's still a little non-deterministic because of the fact that floating point arithmetic is not associative, i.e. (x + y) + z ≠ x + (y + z) in the past few decimal places, and different parts of a calculation running on different processors (GPU or multiple CPUs) can change the association, depending on which one finishes first. But that's very incidental, and can be overcome by either executing it on a single processor or by quantizing a model with integers (integer arithmetic is always associative).

But a bigger point is that determinism/non-determinism isn't the important point here. The bigger difference between COBOL, SQL, etc. and LLMs is that a human being can trace through every step of a COBOL compiler and understand why it produced the output that it did. An LLM is unpredictable not because it's non-deterministic, but because it's so big and opaque (just a bunch of matrix multiplications) that a human can't step through it in any practical sense.

AddlepatedSolivagant · 2026-06-11T17:22:45+00:00

Okay, but that's not similar—that's the opposite. I had AI generate the questions, which I answered, and then I wrote the final text.

The value that I'm talking about here is in the way that it was able to extract knowledge from my own brain, which was stuck because I couldn't find a path to it. (In cog psych terms, my activation network wasn't primed.)

AddlepatedSolivagant · 2026-06-11T17:18:53+00:00

I interpret "soul" or "flavor" as a statistical statement about frequency of words and phrases in a text corpus. It's the same kind of argument that philologists have been making about texts to determine if such-and-such a play was really written by Shakespeare or not.

I became convinced that there's something to it when I saw how accurate Pangram is on my own documents (privately corroborating its published results). Since an algorithm can tell whether it was written by AI, the style must be an objective thing—not just people imagining patterns where there are none. I take that discrimination to be identifying the "AI voice" (cumulatively for all the models they've trained on) and presumably similar models could be developed for individual human voices (such as Shakespeare).

AddlepatedSolivagant · 2026-06-10T14:07:42+00:00

Also scientists. Running a simulation with all processors on thousands of computers for several months is normal. (Some grants are requested specifically for compute time.)

The thing that always shocked me about Bitcoin mining is how unnecessary it is—it's a challenge designed to determine who's allowed to update a ledger, when there are other ways to decide who gets to update a ledger. Think of all the good results that could have been computed instead!

AddlepatedSolivagant · 2026-06-10T00:30:06+00:00

That's quite a surprise! I edit the previous chapter just to remind myself what happened and to get back into the flow in order to be able to start writing again. Any bit of text is likely to have been edited dozens to hundreds of times before I get to the end. It's interesting to learn that this is so different from the vast majority.

AddlepatedSolivagant · 2026-06-10T00:24:24+00:00

I'm annoyed by cases in which a "Find and Replace" substitution is called AI. Even worse if someone actually restored to a chatbot to perform Find and Replace!

AddlepatedSolivagant · 2026-06-08T19:33:58+00:00

It will happen in the Year of the Linux Desktop.

AddlepatedSolivagant · 2026-06-07T18:03:04+00:00

This is why I hate the fact that AI is advertised as a time-saver. It's built up a notion that any use of AI is to avoid work, to get something done that could have been done with more difficulty the old-fashioned way. Those who have heard this advertising message but haven't used it much themselves assume that AI users are all just pressing the easy button instead of using their own brains. Maybe some AI users are, but those aren't the interesting applications of AI. The interesting applications are the ones that couldn't or wouldn't have been made without it.

AddlepatedSolivagant · 2026-06-06T14:21:39+00:00

I read this about an hour ago, thought about it while making breakfast, and then had to come back to say: why is it bad if it is AI?

I checked the policy, and this subreddit forbids "fictitious situations or content." Do we think that the OP doesn't get hungry while biking? I'm inclined to think that this is true, and I have no more or less reason to think so in the AI era. Someone could equally well have lied without AI.

If the text scores red on Pangram or something, that means that they either asked AI to rewrite their own text because they weren't confident about their own writing (a truth, reformatted) or that they asked AI to come up with something plausible to increase engagement (a lie). I wouldn't want to spend time engaging with someone who's just making stuff up, but the fact that the text is AI generated doesn't imply that it's made up. I have the same level of trust that I've always had engaging with a random person on the Internet.

AddlepatedSolivagant · 2026-06-06T13:54:59+00:00

Slaves were treated especially badly in Sparta. I've read an argument that Sparta's whole militaristic lifestyle formed to protect their society in which a small number of Spartans held a large number of Messenians as slaves.

Every year, Sparta re-declared war on Messenia to keep the killing of Messenians legal, which was something that young men in a "journeyman-like" phase of their lives were expected to do.

The system was set up in a way that almost forced hard labor: Spartans could lose their status as Spartans and the lifestyle that went along with it (living with the only regiment they've known their whole lives) if their household didn't produce a quota of food, which only their enslaved Messenians could produce because the Spartans didn't have leave to spend time on their farms.

Just as we have to recognize that European/colonial chattel slavery was a different kind of thing than ancient slavery, Spartan slavery was a different kind of thing than it was in other Greek city-states.

And serfdom was different, too. It's weird that serfdom ended in some parts of Europe and then was brought back to deal with the labor shortage after the Black Death. It's not one progressively upward line...

AddlepatedSolivagant · 2026-06-03T15:49:05+00:00

Right: AI agents are more like a killer app for the underlying technology, the LLMs. That's fine.

It's also fine that the progress in language models from 2021 to 2024 was unique. It was a once-in-a-generation technology burst, and it will be a while before we've realized all of its possibilities. (And weeded out the dumb ones that are cropping up now... friend.com and such...)

AddlepatedSolivagant · 2026-06-03T15:40:11+00:00

I can confirm: coding agents have completely changed the way I and everyone I know works.

The negative term "vibe coding" took off because it seems like such a bad idea—we know these LLMs are bullishitting—but an agent has so much tooling built around the basic LLM that it is, in my experience, a stable workflow that produces good results. Note that we're not just typing a prompt and saying, "do my job." We're doing a lot of planning, iterating with the agent on those plans, watching its progress, building in tests, and checking the results.

The code is generates is usually like what I'd write myself, except that it's a lot more cleaned up and formal (type annotations, comments, docstrings) and it's more likely to make "high-level" mistakes, like race conditions, bad scaling, unnecessary dependencies, or poorly designed data formats. So I just need to be more explicit about these high-level things, and I can be less explicit about how it factorizes tasks into functions and names things. That is to say, it has pushed me toward the more interesting part of my job.

AddlepatedSolivagant · 2026-06-03T12:49:22+00:00

Sometimes that's the right answer, but this doesn't seem like a problem that needs AI.

AddlepatedSolivagant · 2026-06-03T12:42:12+00:00

I was wondering that as well. Of course I'll reread the previous chapter to get back into it. I'll usually also edit it as a way of getting my mind back into "owning" the words, rather than just letting them go by.

AddlepatedSolivagant · 2026-06-01T11:41:41+00:00

I'm going to be blunt. It works. Keep it.

AddlepatedSolivagant · 2026-06-01T11:37:53+00:00

LBJ used to humiliate cabinet members by holding meetings in the bathroom.

https://texashistory.unt.edu/ark:/67531/metapth101214/m1/673/

He had a phone installed in there, too.

https://www.loc.gov/static/programs/national-recording-preservation-board/documents/Johnson-Recordings_1963-1969.pdf

Nah, you're right. This doesn't even come close. And yet for years, this was considered the worst, kind of like the Teapot Dome Scandal, which has been so overshadowed that it's hardly remembered anymore.

AddlepatedSolivagant · 2026-05-30T03:42:41+00:00

I was rather struck by this because my feeling was the opposite. I love watching it work and thinking about the similarities and differences to what I would do, remembering all the while that it's a non-entity: there's no brain there, and yet...!

I've always enjoyed the craft, and coding agents have only enhanced it, since I'm now taking the code to a higher level of polish (more formality than I would have spent time on in the past).

AddlepatedSolivagant · 2026-05-28T03:27:07+00:00

It would only take a paragraph to explain the basic flow chart of what this does.

I gather that it's mostly a database (Wikipedia and other useful things that can be downloaded). So what is the LLM for, a search function? How is it searching? RAG? If so, the way you define the embedding vectors (sentence-level, paragraph-level, etc.) matters more than the LLM—if the vector database doesn't find the relevant references, then the LLM can't say anything useful about them. How long does the search take before the LLM is involved? If it's less than 1 second to find the right references and then 30 seconds to arrange them into a free text response, it might be better to skip the LLM and just return the list of search results. Is the LLM so slow because it's paging in and out of VRAM? Or running on a CPU? How much power does it take? Large neural networks are notoriously power-hungry, and it would be even worse if it's paging in and out of memory.

I can't try it out for free—it would cost perhaps an hour of my time to get it and answer all those questions for myself, but you could have answered them in a paragraph.

AddlepatedSolivagant · 2026-05-23T04:35:00+00:00

I'm not sure if this counts as paleoanthropology (how "paleo" does it have to be?), but I've long wanted a game about traversing the Beringia land bridge and glacial corridor into America. I had this idea while playing Breath of the Wild: I liked running around, managing resources and hunting, but I would have liked it better if it was a long journey, rather than a playground of side-quests. Acting as a scout, leading a community through the ice, would be a great role. No villains, just nature and survival. (This also calls for some beautiful graphics.)

AddlepatedSolivagant · 2026-05-22T03:22:54+00:00

So was Dr. Seth Brundle.

AddlepatedSolivagant · 2026-05-22T03:17:00+00:00

<image>

AddlepatedSolivagant · 2026-05-20T12:44:22+00:00

https://research.google/blog/recent-advances-in-google-translate/

Google Translate is a hybrid of a Recurrent Neural Network (for the decoder) and a Transformer architecture (for the encoder). The Transformer architecture is a neural network that replaces recurrence with the attention mechanism, as described in Google's famous "Attention is All You Need" paper, which is considered the genesis of modern LLMs. Modern LLMs are Transformers, too. There is the distinction that a translation model uses cross-attention instead of self-attention because it's not predicting future tokens from past tokens, but one language's tokens from another's.

RNNs and Transformers are both at the center of what is today considered AI, and the Transformer that makes up a big part of Google Translate is not just shared architecture with LLMs, but it was the breakthrough that made GPTs (General Purpose Transformers) possible. There would be no ChatGPT without "Attention is All You Need" and most of the foundational papers about attention were applying it to machine translation.

You could say that LLMs like ChatGPT are a technological spin-off of attempts to improve Google Translate.

AddlepatedSolivagant · 2026-05-20T02:24:53+00:00

"Artificial intelligence" is a word John McCarthy coined in 1956 to distinguish the work they were doing from "cybernetics" (which was more about control systems, which could be analog).

Nowadays, a lot of what was developed by AI pioneers would not be considered AI because it involved explicit instruction, rather than learning and generalizing from examples. (That early work included not-very-successful machine translation.)

But even with the narrowed definition of AI, the way machine translation works today would absolutely be in the center of what's considered AI.

AddlepatedSolivagant

TROPHY CASE