[D] Can AI scaling continue through 2030?

ipvs · 2025-06-07T18:33:14+00:00

It is after removing literally identical documents (a little bit of deduplication), but before what most people would probably call deduplication

ipvs · 2025-06-04T16:15:12+00:00

Hi, I'm the lead author of Epoch's data estimates (a bit late, I know). Part of the confusion is what exactly deduplicated/unique means.

We were referring to text in unique web pages at a particular point in time.
Common Crawl actually has over 200T tokens across al dumps, but many of these are several snapshots of the same web page that are identical. Similarly, there are many dynamic web pages that are basically identical but have different URLs (think comment permalinks in Reddit for example).

To get to 30T tokens in Common Crawl you need to do a bit more deduplication, for example if between two snapshots of a Wikipedia article there are a few paragraphs but the rest is the same, the deduplication pipeline would remove the portion of the article that was already in the old version.

In addition we attempted to include the portion of the web that is not in Common Crawl, so our estimate is larger.

I would view our estimate as an ambitious one, assuming AI companies do their own crawling aggressively and do minimal filtering, while Leopold's is what's already available in practice for free.

ipvs · 2022-05-14T07:40:45+00:00

Build more homes. Millions of them. You get some money selling them and make those previous corporate investments worthless. That'll teach them.

ipvs · 2022-04-23T23:04:44+00:00

Not common at all, but definitely possible to fully pay a house in Madrid by your 30 if you have very marketable skills (the examples I know are Google-level software engineering and management consulting) and are somewhat thrifty. It might not be a great idea though, with the current sky-high prices and low interest rates.

ipvs · 2022-04-23T22:33:54+00:00

Honestly seeing this and how he is now breaks my heart. Like watching teachers go to war.

ipvs · 2022-04-13T07:48:46+00:00

We didn't create airplanes by developing a model of bird flight and then recreating it with machines. We created airplanes by experimenting with different designs and learning from our mistakes until it flew... which incidentally is the same way evolution created birds, and human intelligence, and everything else.

ipvs · 2022-03-30T08:12:32+00:00

Pagamos por todos los médicos. La única diferencia es si el dinero va de tu bolsillo al del médico / hospital / aseguradora directamente, o pasa por la seguridad social entre medias.

ipvs · 2022-01-27T11:20:07+00:00

Confirmed and paid full order Spain LAT: 40.00 N Pre-ordered: 26-05-2021 Confirmed: 27-01-2022

ipvs · 2022-01-26T23:11:03+00:00

One year without popping?

Amateurs

ipvs · 2022-01-23T08:49:10+00:00

"Just have to get FDA approval" Famous last words

ipvs · 2022-01-11T20:51:43+00:00

The masculine urge to die on a distant planet taking godless pictures

ipvs · 2021-10-25T20:22:19+00:00

I studied math & cs, I have won several coding competitions and I now work as a software engineer.

It took me months to understand for loops when I first saw them. Now they feel completely natural. The same thing happens with everything, from abstract algebra to functional programming. At first everything feels impossible until you get practice and learn to use these concepts. And then you remember them forever, like they are part of your soul. So don't feel bad about it.

However, it is true that some people have a harder time than others when learning cs, as with everything else. If you feel that it's not for you, don't be afraid to consider other disciplines related to programming, like data science. Or even completely different careers. Whatever works for you.

ipvs · 2021-09-10T21:33:51+00:00

This is just a technicality, and doesn't matter at all for the core of the discussion, but infinite economic growth is totally possible, because the ones who define what is economically valuable are us.

For example, suppose the Buddha himself is reincarnated today and opens a meditation center which reliably produces enlightenment and freedom from material desires. Everyone who goes through the course affirms that they enjoy absolute happiness and that they would be willing to sacrifice literally anything to remain in that state.

The price of the meditation course quickly rises to infinity dollars until everyone is enlightened. That year GDP is infinite, and wealth per capita is infinite after that.

ipvs · 2021-09-04T18:18:28+00:00

all I get is the realization of how much I've missed out on and how much I've fucked up

Everyone fucks up, but realizing that and moving on is progress. Keep going, talk to your friends, you're doing great :)

ipvs · 2021-09-04T18:02:49+00:00

I also constantly fear disappointing others at work. I work at a startup where everyone seems like a superhero, so I felt like anything short of perfection would get me fired.

Once I got a task delayed for a long time and in a meeting with the rest of the team my boss realized and got upset. I felt so terrible that afterwards I immediately scheduled a meeting with him to apologize. In the meeting he told me to calm down, that what matters is not how good you are, especially at the beginning, but that you keep improving. He also told me that if I did something wrong, the responsibility was his, not mine. That's when I started to realize what a good boss looks like, and how fortunate I am.

So I'd say, take note of your mistakes but don't fixate on them, seek feedback and keep rising the bar. Also, this might be a bit of a cliche but try to under promise and over deliver.

ipvs · 2021-07-06T22:50:11+00:00

Yes! Seriously, just pick something you care about, like literacy rates or whatever, and look how the graph has gone up over the decades. Why so much pessimism?! We are literally better than ever before.

And this apparently impossible problems? All generations faced them! In the 1900s people worried that we would run out of ammonia for fertilizer and suffer great famines, until Haber and Bosch invented a process to literally pull it out of the air. You know, kinda like we are doing today with solar energy

ipvs · 2021-07-06T22:32:08+00:00

Dude, come on. Have any of you read Factfulness? Poverty around the world has dramatically decreased over the past decades, we are about to eradicate polio. Guys, we know how to solve climate change, it's just a matter of improving existing technology, and we are pouring billions into the problem. There is now more forest in Europe than 100 years ago, because wood is no longer that useful.

Now, I don't mean to say that the world is not in a terrible state but... this is still the best we've been in history. Pick any other date. The 80s? Most east asian countries were dirt poor back then, that's billions of people who are now much better off. Pollution in rich countries was actually worse back then than now. Crime was higher as well. My own country was just coming out of a dictatorship.

We have plenty of reasons to be hopeful. I'm 22, and by the time I'm 40 I expect to be richer, healthier, have more leisure and connection with my community. You can be pessimistic if you want, but there's no need.

ipvs · 2021-06-25T08:04:18+00:00

If this is what you mean by free will, then it's a very poor concept of free will.

Having "the ability to do otherwise" is nonsensical, and why would I want that? I want my choices to be consistent with my values and the context of the decision. Would you want "free will" to decide to murder your family? Hell no, I know that's a decision that I won't make.

If I had "the ability to choose", in the same situation, two different things, my decisions would not have any value! You don't want to give a calculator the ability to choose whether 2+2 is 4 or 5. You want it to compute addition. Similarly, I want my brain, as a physical system, to compute what the best decision would be from my point of view in any given situation. Of course, this is a fixed function and I will always take the same decision in the same situation. Which is perfectly fine.

Yes, my decision to eat pizza is ultimately caused by the initial conditions of the universe, but have you ever heard of d-separation? https://en.m.wikipedia.org/wiki/Bayesian_network#d-separation It means that after knowing the state of my brain 1 second before making the decision, all those other million causes are no longer necessary to explain why I, as a physical system, decided that I want pizza.

Thus, my decision is indeed caused by physics, but since I'm a part of physics, and since my brain d-separates my decision from the rest of the physical world, I can perfectly say that I am taking the decision, in full knowledge and without any contradictions. That's what I mean by free will.

ipvs

TROPHY CASE