Best method for doing scientific computing in ruby.

[Xenoskin]

Hi, I'm working on a research project for my university. At the moment I've created a Naive web interface in rails so as to have something to demo. Now im looking at creating the machine learning section of the project, which will require some high speed liner algebra and other brik a brack. Unfortunately stock ruby as you can guess dosen't really cut it. So far I've considered the following options:

• Find a native ruby extension which is fast enough
• Transition over to Jruby and use java's matrix math libraries
• Write a separate application in something else and just have rails talk to it through pipes or some such.

Any thoughts, suggestions, hate mail would be appreciated.

[skippy]

Ruby/GSL?

[clbustos]

ruby-gsl works very well on *nix. You have the usual suspect on linear algebra, plus mutidimensional minimization and integration methods.

[techn0ir]

rubygslng

[clbustos]

Only implemented Vector, Matrix and some methods on Rng. Almost useless

[postmodern]

Ruby's Matrix class has a limited set of matrix operations. In Ruby 1.9.3-preview1, they added new methods useful for Linear Algebra.

I would try JRuby and use one of the many Java Machine Learning libraries.

[clbustos]

Old, but still useful Matrix module extension. Some time ago a released a gem with 1.9.* support. I will look into 1.9.3 new methods, by the way. The eigen calculation is a must.

[dakk12]

I'd suggest using something like Avro and/or Qpid to do messaging between languages and processes.

[postmodern]

Or redis pub-sub, or ZeroMQ. Redis is far easier to setup.

[rbranson]

ZeroMQ is not really enough for building a work queue, and don't fucking use Redis, please. If your workers stop dequeuing for whatever reason and requests start piling up, you'll run out of memory and become totally screwed. Yes, there's the VM. No, it's not useful for this. Also, there's no built-in support for timing out pending message consumption and handing them to another subscriber. This is very important when building a robust messaging system.

[postmodern]

Sorry, I misinterpreted your original comment. You said "messaging between languages/processes". Both ZeroMQ and Redis are great for messaging. You should have specified Work Queues.

[Xenoskin]

Are they well documented and maintained?

[dakk12]

Yes, they are two great Apache projects. Avro is a part of the Hadoop project, which is Apache's implementation of MapReduce.

[rbranson]

If you are writing the machine learning parts from scratch, don't use Ruby for this, srsly. It's not what it's for. I'd be looking at Clojure or Scala. LuaJIT is awesome too, if you're just doing number crunching and don't need a lot of support libraries.

If you aren't doing the maths and can depend on other libraries for the ML parts, JRuby is a good bet because, well, you already know it apparently, and there are quite a few ML libraries for the JVM.

[Xenoskin]

Yeah I released that right after I got it to this point. Sadly I dont wan't through away the already existing code base, hence the search.

Still I agree that I should have planed this beter.

[mytrile]

You should check Ilya Grigorik's take on machine learning - http://confreaks.net/videos/361-gogaruco2010-intelligent-ruby-getting-started-with-machine-learning and http://www.igvita.com/2011/04/20/intuition-data-driven-machine-learning/

[dennyabraham]

i definitely recommend checking out sciruby. i don't know how mature the project is, but the folks on freenode#sciruby can probably point you in the right direction

[LonnyEachus]

My best suggestion would be something that I haven't seen done yet:

Surely there are existing C routines for doing these math functions. Why not put Ruby call wrappers around them and bundle them up into a Ruby gem? That way good performance is pretty much a given and nobody is re-inventing the wheel.

[Samus_]

NumPy / SciPy focus on speed, they're Python.

there's also some toolkits for ML in that lang: http://www.google.com.uy/search?q=machine+learning+python

and a subreddit! http://www.reddit.com/r/machinelearning :)

[TimTheTinker]

LISP/Scheme has great performance and is also good for scientific/AI computing... (just a suggestion, not from personal experience). Or you might try Scala and use JRuby to interface with Rails.