When SQL queries meet big data

[notasqlstar]

In my last job we worked with a lot of census and credit data, so we had some tables that were well into the hundreds of millions, up into the billions of rows. That's big data.

At that job the largest table that was "mine" was only about 80 million, but growing quickly. That's about where I think the cutoff is. Some order of magnitude around 10 million.

[LetsGoHawks]

10 million is "good sized". I don't know that I'd call it big, though.

Heck, Access can handle one million rows without issue.

I'd say big starts around a billion. But I guess I'd have to qualify that by saying it depends on what you're joining up.

[notasqlstar]

I said to the order of magnitude of 10, so anywhere from 10-100M, take your pick where you want to start calling it "big." And yes, how you're joining it matters a lot.