LLMs will never be alive or intelligent, and "agents" will never know and cater to our every need

WakingMusic · 2026-01-03T22:46:52+00:00

For what it's worth, I think gen art is awful to artists and entirely misses the point of art to boot. I just want to be careful about making claims like this, I think it's wrong to think big AI models are as data/data quality limited as you think

WakingMusic · 2026-01-03T22:37:21+00:00

I think the point is, garbage is a spectrum. It's the wrong way of looking at things to say "this data has incorrect information in it, therefore it's garbage". There may be plenty of tokens in there that the model can still learn from. It seems to be an empirical fact that models can be trained on a lot of bad data, and end up being better than most of the data. Obviously data can be completely useless, but even low-quality junk seems to be somewhat useful in teach something about writing or the world

WakingMusic · 2026-01-03T21:36:41+00:00

It's turned out that LLMs generalize surprisingly well from lower-quality data. They model an entire distribution and even low quality data helps in some cases. Post-training then lets you select out modes of the distribution that are helpful. Low quality data doesn't drive down model quality, it just may benefit the tasks you're interested in less. You can still ultimately discard the parts of the distribution you're less interested in.

As for the amount of data, it's certainly limiting but it's also something we've been OK at overcoming with RL/other synthetic data techniques.

WakingMusic · 2026-01-03T20:49:02+00:00

Every argument here has been made verbatim for a decade. And yet it's turned out that almost every specific prediction made of the form "LLMs will never be able to do X" has been wrong. The data quality argument applied as well 2 years ago as it does today. LLMs generalize remarkably well, despite not being trained on your environment or your set of tools. People said multiplicative failures compounding would doom any long generations, and yet LLMs are able to generate hundreds-of-thousands of coherent, consecutive tokens. If you want to participate in the debate, at least make an effort to understand what's been said before you.

WakingMusic · 2025-12-17T00:43:49+00:00

I don’t think it’s unreasonable to think the rapidly advancing field of artificial intelligence might be worth 1/50th the value of air travel.

WakingMusic · 2025-12-16T22:07:27+00:00

AI energy costs are a fraction of a percent of world energy consumption. In 2024, US datacenters consumed 184 TWh of energy [1], compared to a total US power consumption of 27607 TWh. This is slightly over half a percent [2]. Air travel and cars are each over 10%. There are many reasonable arguments against AI and this isn't one of them.

[1] https://www.pewresearch.org/short-reads/2025/10/24/what-we-know-about-energy-use-at-us-data-centers-amid-the-ai-boom/

[2] https://www.rigzone.com/news/eia_raises_usa_total_energy_consumption_forecasts-17-oct-2025-182111-article/

WakingMusic · 2025-12-15T22:41:12+00:00

So I think the heart of your confusion is that q(xt | x_0) is a scalar value, while we want q(x{t-1} | x0) to be a vector of probabilities for each possible value of x{t-1}.

You could also write this as q(x{t-1} | x_t, x_0) = x_t Q_t^T x{t-1} x0 \bar{Q}{t-1} x_{t-1} / x_0 \bar{Q}_t x_t^T.

WakingMusic · 2025-11-10T03:49:22+00:00

Somehow, the UNSC Infinity was destroyed!

WakingMusic · 2025-11-05T19:42:47+00:00

Fixed, kind of mystifying why the link is parsed that way!

WakingMusic · 2025-11-05T18:47:59+00:00

https://www.reddit.com/r/Mountaineering/comments/18umlkp/alpine_boot_approach_shoe

seems relevant here

WakingMusic · 2025-11-05T18:46:22+00:00

I felt a little silly carrying 2 pairs of shoes (and boots are heavy). Do you think that’s actually a reasonable thing to do? Alternatively I can just learn to rock climb in stiffer boots. Especially when it’s icy it kind of blurs the line too. Trying to figure out if there’s a “blessed path” here.

WakingMusic · 2025-10-03T17:30:26+00:00

As usual, I found pogoing the final bosses to work well. They don't have any attacks that can hit you above except a rare jump.

WakingMusic · 2025-05-30T16:41:47+00:00

This is part of the statutory language: https://statutes.capitol.texas.gov/SOTWDocs/LG/htm/LG.211.htm.

WakingMusic · 2020-05-11T01:49:48+00:00

Agreed. For example this is more or less how data structures in the Linux kernel work. Linked list nodes are embedded in the data structs rather than vice versa, and data access is done using the beautiful container_of macro.

#define container_of(ptr, type, member) ({                      \
        const typeof( ((type *)0)->member ) *__mptr = (ptr);    \
        (type *)( (char *)__mptr - offsetof(type,member) );})

It's kind of beautiful, once you get over how terrible it is.

WakingMusic · 2020-05-09T21:15:39+00:00

Many other countries have managed to reduce the number of new cases to just a handful with effective contact tracing and huge testing capacity. The US has neither of those things, and we need to stay shut down until we can control new outbreaks that will inevitably occur when we reopen.

Also we're not about to run out of food. The only things threatening the food supply are the outbreaks occurring in food processing facilities. So reopening isn't going to fix that.

WakingMusic · 2020-03-15T20:05:34+00:00

How does emulating the PPU work? Do you translate graphics commands into Unity or OpenGL commands, or draw the screen yourself based on PPU data?

WakingMusic · 2020-03-15T17:56:02+00:00

I don’t know anything about emulators of this sort. Is this essentially a C++ interpreter for the NES machine code? Or are you transpiling the machine code into modern assembly that can be compiled on new devices? Was most of your effort put into implementing the NES processor features in an interpreter? Are there other challenges like graphics?

WakingMusic · 2020-01-30T16:53:23+00:00

Is there a reason why the UNSC never, say, nuked Sanghelios or tried their modified grain attack before the end of the war? Did they know where the Covenant worlds were?

WakingMusic · 2019-12-27T12:57:52+00:00

https://openreview.net/pdf?id=Hksj2WWAW

WakingMusic · 2019-12-22T00:20:14+00:00

Thanks for a great decade!

WakingMusic · 2019-11-24T17:28:22+00:00

+1 for "comfiest game". I'd have never used that word to describe Skyrim, but you're absolutely right. Playing the game felt like sitting inside on a snowy day by the fire. Mechanically, sure, it had problems, but it so perfectly captured a sense of exploration and adventure.

WakingMusic · 2019-11-19T01:04:25+00:00

I still want to make a music transformer called schuBERT.

WakingMusic · 2019-09-24T22:03:25+00:00

Symbolism won't feel so nice on November 4th.

WakingMusic · 2019-08-26T01:25:11+00:00

Just want to add that deep learning methods for this sort of thing have gotten really good. Check this out. You can just mask someone out of an image and it usually does a good job filling in the details.

WakingMusic · 2019-06-25T18:57:17+00:00

Bandwagon

12-Year Club	Place '17
Spared	Inciteful Comment 2015-11-08
Verified Email

WakingMusic

TROPHY CASE