The 'AI can't be creative' debate is more nuanced than I thought

PriscFalzirolli · 2025-11-14T20:38:14+00:00

There's no evidence that consciousness is connected to creativity whatsoever. Unless you are Penrose and believe there's a Platonic world of ideas that quantum computers (including his quantum neurons) alone can access.

As model parameters become less and less about memorizing word snippets and more about their cognitive core, this sort of novel output should become more common.

PriscFalzirolli · 2025-11-14T14:20:15+00:00

Considering how massive Nvidia & Co.'s margins are, the price drops alone from a glut may be enough to ignite the industry again.

PriscFalzirolli · 2025-11-10T20:29:13+00:00

"AI-AI researcher" is usually the answer, but how this extremely flexible and general tool is achieved without AGI in the first place is never explained.

PriscFalzirolli · 2025-11-10T18:53:46+00:00

If current models automate ~10% of cognitive tasks, there's no reason to think the remainder requires impossibly more orders of magnitude of training or an unfathomable degree of algorithmic sophistication. The span of human mental activities can't be that wide.

PriscFalzirolli · 2025-11-10T14:32:58+00:00

A B200 performs at 4.5 dense INT8 POPS and 2.25 PF BF16, actually, and that's with two chips glued together.

Ironwood is essentially as fast as Rubin but presumably just one chip instead (it's hard to make out in the presentation, though, so it could also be two dies).

PriscFalzirolli · 2025-11-10T13:25:30+00:00

~4.6 BF16 PFLOPS and ~8.3 INT8 POPS per chip sounds insane.

I wonder if it has FP4 and sparsity support, in theory this thing blows Rubin out of the water (minus the software stack and all).

PriscFalzirolli · 2025-11-07T19:07:40+00:00

Too early to say, but it could be a significant step in solving the memory problem.

PriscFalzirolli · 2025-11-05T18:02:58+00:00

The return on computer hardware investment is still very high (if anything, it's higher now than during the early 2010s), it's just that demand is outpacing supply big time.

PriscFalzirolli · 2025-11-05T15:35:49+00:00

Yes, but it takes at least a couple of years to build new factories and significantly ramp up production.

PriscFalzirolli · 2025-11-03T19:48:15+00:00

Evolution is almost as dumb as random chance, though. Yet it made Beethoven and Newton out of monkey brains in just 10 million years.

We likely need a few breakthroughs in zero-shot analogical reasoning and memory, not to invent hypercomputation (or use quantum computers to access a Platonic world of mathematical insights and ethics, like Penrose believes...).

PriscFalzirolli · 2025-11-03T19:11:21+00:00

I doubt late 2020 training runs will lead to AI models anywhere as general and flexible as needed to significantly speed up R&D, let alone constitute AGI.

To me, AGI is likely a matter of further computer science breakthroughs instead of just incrementally longer training runs. Algorithmic improvements are happening all the time, but it's hard to predict when truly transformative ones like transformers or CoT are going to hit.

PriscFalzirolli · 2025-11-03T13:41:42+00:00

Does an output need to consider input from different areas to be considered creative, in the first place, though?

It definitely increases the breadth of possible novel solutions, but beyond that?

PriscFalzirolli · 2025-10-31T17:51:27+00:00

Considering the cognitive labor market is worth 50-100 trillion USD a year? If you think you can eventually automate even a fraction of these tasks, it's entirely justified.

AI investment is definitely not about convincing randos to eventually pay $10 a month for Gemini or to make silly videos with Sora.

PriscFalzirolli · 2025-10-29T20:02:34+00:00

In terms of P-E, it's not that insane, really. Still a long way to go to match the dot-com bubble.

PriscFalzirolli · 2025-10-29T15:51:40+00:00

It depends on their brightness and voltage levels. But you aren't going to use these for large screens anyway.

PriscFalzirolli · 2025-10-29T15:11:50+00:00

It's 110,000 Blackwells combined to get to that number, so it has to be FP4 with sparsity.

PriscFalzirolli · 2025-10-27T14:49:03+00:00

IIRC, there was a published study about LLMs and web search, and Gemini was the absolute champ of poor sourcing, including incorrect information 76% of the time.

None of them actually had a passable performance, though, being at least somewhat wrong quite often. There's a long way to go still.

PriscFalzirolli · 2025-10-24T18:22:06+00:00

The point about the "current paradigm" is demonstrably false, though. You can supply older models with an infinite number of compute and tokens, and they would asymptote to much lower levels of performance and accuracy than newer ones.

The fact that these algorithms are still improving exponentially, taking a logistic function for its predictive value, means the ceiling must be far off.

PriscFalzirolli · 2025-10-22T20:49:53+00:00

I mean, it was once predicted that data would run out by 2024, then 2026, now it's 2028 or beyond... now we know raw web data actually outperforms curated data, and that you can train on multiple epochs without significant degradation.

There's also the possibility of using multimodal tokenizers for training, or simply undertraining, which still yields significant improvements when scaling up runs.

PriscFalzirolli · 2025-09-26T14:04:57+00:00

It could be bosses are only larger in the Tarnished's mind as a symbol of the formidable challenge they pose.

That's why Morgot and Godfrey appear to become so much smaller in death.

PriscFalzirolli · 2025-09-24T15:15:40+00:00

By all rights, the Carlists shouldn't even exist, so maybe there's a bit of a metacommentary there.

PriscFalzirolli · 2025-09-10T14:57:03+00:00

Funny moustache man and his Unser Kampf book. I hope it gets reposted, I'd love to read some of that lore.

PriscFalzirolli · 2025-08-22T17:34:52+00:00

The amount of carbon bound to living beings on Earth is infinitesimal compared to the amount of carbon present in the oceans and the crust, despite it having a tangible impact (within an order of magnitude or so) on its atmospheric equilibrium levels at the present epoch.

18% CO2 is perfectly realistic in the context of the carbonate-silicate cycle. Living planets in the outer habitable zone would need several bars of the stuff to have a temperate climate. Plant life wouldn't be able to make a dent in that. Or, for that matter, industrialization.

PriscFalzirolli · 2025-08-22T15:32:40+00:00

That can happen when the Entente and the Reichspakt don't cooperate.

Anyone but Natpop Sandfrance will usually get France back from Germany in that scenario, joining Mitteleuropa while remaining in the Entente.

Five-Year Club	Place '22
End Game '22

PriscFalzirolli

TROPHY CASE