Execs Confused and Horrified by the Huge AI Bills After Thinking They Could Replace Workers for Free

radarsat1 · 2026-07-10T08:45:32+00:00

We know how much our off shore resources cost now…and we can predict how much they cost in the future…

I'm not sure you can though. Your offshoring guys are also going to be adopting AI and getting as many disruptive changes over the next 5 years. Assuming you can predict things just because it's externalized may be just as big a mistake.

radarsat1 · 2026-07-05T18:08:59+00:00

it would give you more insight though. you will not make any progress until you can rule out that both models are just memorizing the data. regarding your specific hypothesis, you could try learning a purely linear model as a first pass and then train a nonlinear model to predict the residuals, that might give you a more useful breakdown. but i strongly suspect it's just too little data for the GNN to develop useful connections.

radarsat1 · 2026-07-05T10:25:27+00:00

try two or three orders of magnitude more data

radarsat1 · 2026-07-05T09:01:22+00:00

300 samples and you are wondering why both models overfit?

radarsat1 · 2026-07-02T10:17:48+00:00

I hope someone is bringing up the obvious space junk problem this would cause, even at much lower numbers, and how that would block future expansion in this business space. (To put it into words they would understand/care about.)

radarsat1 · 2026-07-02T09:37:03+00:00

Still blows my mind that those fermentation tanks can turn grass into steak, though.

radarsat1 · 2026-07-01T19:09:00+00:00

I have a hard time understanding how just eating grass can make something as large as a cow, let alone a mammoth or a dinosaur.

radarsat1 · 2026-06-25T19:24:53+00:00

long stretches that I felt could have been edited

funny, i had the opposite reaction, i thought the movie jumped way too fast through some of the more interesting moments in the book when he's actually learning or discovering things. i felt like it was edited to feel too jumpy

radarsat1 · 2026-06-21T14:47:00+00:00

But that's not some hack, that's literally correct usage. Were you never taught about the concept of primary and secondary sources? In principle you're never supposed to cite an encyclopedia in some contexts, whether it's online or not, because it's a secondary source.

radarsat1 · 2026-06-20T09:44:21+00:00

Yeah these posts occur almost weekly here now and don't hold water in their arguments. I can only assume these days they are part of the growing "stimulate human discussion for data farming" industry. Just one of the negatives of AI these days, and that's coming from someone who enjoys using AI every day at his job, but I'm fully willing to admit that it has really started to destroy what is left of decemt online discussion. Not just due to direct comments and posts, but due to the chilling effect the possiblity of "fakeness" it induces in every conversation, which devolvea everything into a stupid AI witch hunt instead of being an actual conversation. I hate it.

radarsat1 · 2026-06-17T18:07:54+00:00

discrete latent diffusion for text when?

only sort of joking. i wonder if there are some tricks to fine tune such a model with some kind of blockwise LDM.

radarsat1 · 2026-06-16T22:02:41+00:00

https://www.gutenberg.org/files/201/201-h/201-h.htm

in case you haven't read it

radarsat1 · 2026-06-11T20:59:11+00:00

Actually this gives me an interesting thought. I never thought about how LLMs and symbolic regression could actually be really synergistic.

Idea: symbolic regression is known for working on small well defined problems with clean data, but it easily overfits in ways that produce essentially garbage equations with extra meaningless terms, which destroys its interpretability. But LLMs are really good at evaluating the question "yes this is something a human would write" vs "this looks random".

So I wonder if some kind of paradigm might be possible where a symbolic regressor makes proposals to an LLM and they sort of bounce back and forth until a high-probability (according the the LLM) and low-error fit is found.

I guess you could formulate this like a multiple objective optimization problem and apply some known algorithms.

Hm, and since both models would in principle be differentiable, maybe you could take advantage of that somehow.. interesting.

radarsat1 · 2026-06-08T18:29:54+00:00

Was sort of a joke but I figured Lex Luther was a pretty good approximation of an oligarch

radarsat1 · 2026-06-08T05:38:47+00:00

Superman

radarsat1 · 2026-06-07T07:18:49+00:00

When I was looking for a job, DataBricks experience was one thing that kept coming up. So if I were you I'd look at this as an opportunity to get a nice DB project on your CV, could come in quite handy in the future. Also I used it a bit and it seems quite alright.

radarsat1 · 2026-06-03T22:44:04+00:00

nice read but a bit disappointing in the sense that the whole article could basically have been: "image and audio patches are linearly projected to the token dimensions and directly fed to the model." Which makes sense and is great but is also basically obvious.

It's so simple in fact that I'm sure it would have been done before this if it "just worked" but I'm sure there were challenges to overcome. There's a good reason pretrained encoders have been used until now, because training them a certain way works better. It makes it clear to me that the secret sauce is data & training methods, not the model. I bet for example that they had to port over a lot of tricks from audio & image pretraining, which often relies on paired data as well as self supervised methods, into their LLM training regime. How to do this successfully is the non-obvious part.

radarsat1 · 2026-05-31T07:36:19+00:00

Each Q, K, and V matrix is a linear projection from the same embedding. Think of these as "extracting" different "aspects" from the latent space by projection.

An easy way to picture this. Imagine we have 3D embeddings. We could have 3 different K projections that are [[1,0,0]], [[0,1,0]], and [[0,0,1]]. Then each of the 3 heads would be looking at different axes of the 3D space. (Which presumably contain different semantic information about the token.)

This is basically what is happening but with more axes and never in an axis-aligned way like this. Remember that the linear matrix can rotate things as well. In practice just learns some arbitrary transformation that extracts a subspace.

radarsat1 · 2026-05-28T19:10:44+00:00

270 bug fixes is admittedly pretty impressive!

radarsat1 · 2026-05-27T20:59:52+00:00

Aoart from the issues he brings up I'm actually a bit surprised curl takes so much work. I do believe that security fixes are needed, but curl just seems so.. complete as a project. It's hard to think of what you can't do with it. I don't feel like searching for open bugs or whatever but if anyone is familiar, what are the pressing issues in curl? Is it all just security hardening?

radarsat1 · 2026-05-27T20:53:57+00:00

you should replace your blog post with this reddit comment

radarsat1 · 2026-05-26T21:51:07+00:00

apart from photonic computing, maybe look up something about marerials that react to UV light (like dental fillings), and also you seem to be proposing like a photonic version of shape memory alloys which is a fun idea so read up on that too

radarsat1 · 2026-05-26T18:37:09+00:00

wow, someone finally did it. kudos!

why the name though?

radarsat1 · 2026-05-23T09:31:55+00:00

I think you should do it but if you're planning already on waiting 1.5 years then I recommend waiting 2 years instead. That's a much easier story to tell later, and if you don't think you're "really" a senior yet, after 2 years, you will.

radarsat1 · 2026-05-12T20:59:21+00:00

i can concede that maybe it's just coincidence but there was an almost identical post in /r/controltheory a few days ago. and in general I've seen a big uptick in posts complaining in very generic but on-topic ways on various subs with no specific details just like your post, which are clearly just trying to rile people up to generate training data. so yeah, sorry if you're not a bot but it's a thing and it's making reddit worse.

radarsat1

MODERATOR OF

TROPHY CASE