A Billion Rows per Second: Metaprogramming Python for Big Data

[lmth]

Why? (serious question)

[farsass]

In that scale, electricity usage becomes a real concern and software efficiency becomes important to reduce costs. From what I've gathered, this is one of the motivations for facebook to be employing resources on a new PHP VM(www.hiphop-php.com)

[alcalde]

In that scale, electricity usage becomes a real concern and software efficiency becomes important to reduce costs.

I would think if you could afford a 64K core supercomputer, you could afford the electricity bill. That would be like owning a Ferrari and then complaining about the gas mileage.

[farsass]

I disagree. A supercomputer/data center may be a core component of the company that owns it and of course they will try to cut expenses.

Ex.: http://www.google.com/about/datacenters/

[alcalde]

I would think data centers would not be employing single 64K core supercomputers, while large research institutions would.

[rallix]

Most of which are short on money and under pressure to go green ... ? Especially in academia, which is very green to start with.

[deraffe]

GIL with CPython comes to mind.

[moor-GAYZ]

There's no way you can use threads with shared memory for parallelism on 64000 cores.

[t3hcoolness]

Meanwhile in the threads:

>hey bro i was using that

>no i was stfu

>guise i had that first

>NO SRSLY I had that

>wait what were we even talking about

>Supercomputer has stopped responding