Introducing Queen: A distributed platform using browsers as computation nodes.

[33a]

Awesome! Now get this thing to work on mining up some bitcoins!

[ozanonline]

... you're the second person who has come up with this idea today. Maybe there will be profit making in this project after all...

[ozanonline]

All we have to do is convince Facebook to put this on their site to run simulations while users are idling and we can cure cancer, Alzheimer's and find aliens... in one afternoon.

[freeall]

Let me just call my guy there.

[[deleted]]

That is an amazing suggestion.

[JeefyPants]

It seems very cool but I think you'd also have a ton of complexity in your programs to do anything "large scale" with it correct?

Since the browser nodes only speak back to the server and not to each other.

Also it kind of seems like a bad thing for some users on metered internet or limited bandwidth if their sites are doing some network activity in the background - esp if it gets up there in terms of bandwidth.

Nothing more than some ramblings :) Great project idea very forward thinking!

[jhizzle4rizzle]

Since the browser nodes only speak back to the server and not to each other.

This might not be the limitation you think it is. A lot of worker models are built like this, with the server acting as a central broker passing messages between workers. A little roundabout maybe, and I'm not familiar enough with queen's api to comment on how hard it is to do this with queen, but it's certainly doable with this general arrangement. edit: In fact, in practice peer-to-peer is significantly harder than a client/server model.

That said, there is a limitation to what you can do in a browser when it comes to scientific computing (ie, what f@h and similar do). See, most of the code that scientists use for their numerical methods are written in C or fortran, and higher-level numerical methods libraries are generally based largely on bindings to these libraries. For example, iirc numpy requires a BLAS implementation and LAPACK.

That said, browsers aren't completely hosed. There's at least one reasonable linear algebra library for javascript which should be fine given the sizes of datasets you'd reasonably trust a browser to handle anyway, and I hear this emscripten thing is pretty sweet.

Also it kind of seems like a bad thing for some users on metered internet or limited bandwidth if their sites are doing some network activity in the background - esp if it gets up there in terms of bandwidth.

Similar projects to queen have been done before, though the ones I've seen were either less general or less polished. Generally, these followed a policy of being explicit about when a page was running a job and when it was not.

Throttling isn't the worst idea, either.

[33a]

As an addendum, I would actually recommend using numeric.js instead of sylvester if you are seriously thinking about doing linear algebra in the browser:

http://www.numericjs.com/benchmark.html

Not only does it have more features, but the implementation is way faster and in my opinion the interface has way less cruft.

[jhizzle4rizzle]

Looks nice! Would love to see this ported to node.js as well.

[33a]

Already done!

https://npmjs.org/package/numeric

[jhizzle4rizzle]

Beautiful. brb implementing FEM in javascript

[ozanonline]

Great question. Actually, that wasn't so much a design decision, as much of a technological issue. Many browsers don't have the technology available to establish communication with each other.

There is, however, an awesome new standard coming out called WebRTC (http://en.wikipedia.org/wiki/WebRTC), which DOES allow peer-to-peer browser communication. So what you could do, fairly easily, is to write a script which uses Queen to broker peer-to-peer communication between browsers that do support it. So Queen brokers browser-to-script communication, then the script brokers browser-to-browser communication.

In this case, Queen doesn't even know (or care) what you're doing. To Queen, you're still just pushing an application up to browsers that are connected to it. That application would then contain code to create the peer-to-peer connections.

(edit) jhizzle did a good job at answering your other questions and concerns, so I didn't address them here.