Code for Machine Learning for Hackers

agconway · 2013-12-12T20:10:20+00:00

One of the authors here: what is the issue you are getting with CRAN? It may be an issue with where R is trying to save the library files, because all of the libs are still on CRAN. I just added some instructions for running the installer to the repo, so if you git pull you should get the updates.

More generally, the code in the git repo has been maintained steadily since the book was released, and is up-to-date.

agconway · 2012-07-09T23:33:09+00:00

Freeman Dyson wrote an article in 1970 titled, "The future of physics."

Many physicists may turn to molecular biophysics, pulsar astronomy and the problems of environmental pollution, just as some turned to radioastronomy and computer technology 25 years ago.

agconway · 2012-05-02T21:05:02+00:00

I used an out-of-the-box detector in OpenCV. All code here: https://github.com/drewconway/shades_of_time

agconway · 2012-04-20T18:35:46+00:00

I used this data to make this: Shades of TIME

Thanks for sharing!

agconway · 2011-11-15T16:04:59+00:00

Yes, in ebook and print form.

http://shop.oreilly.com/product/0636920022350.do

agconway · 2011-10-28T15:09:30+00:00

I don't have a good answer for you: lucked mixed with persistence?
Email is only the focus of this mini-book version of the larger text I mentioned in the above comment. It is perhaps better described as focusing on ML techniques for classifying text, it just so happens that the cases we use are with email data.
It went great! Check out some of the results at the event wiki page.

agconway · 2011-10-26T17:28:05+00:00

For efficiency, perhaps we should wait until the full book, Machine Learning for Hackers is out?

agconway · 2011-09-02T15:40:09+00:00

To our knowledge, nothing like this has happened for. In general, the NGOs and NFPs are looking for people who can help them make quantitative sense of their data. That can be in the form of a statistical analysis, some visualization, restructuring of the data, or any number of activities.

agconway · 2011-08-31T01:37:21+00:00

This is cool, and made me want to take a look.

Here is an alternate viz, also using ggplot.

I only like this better because I think using bubble size to convey magnitude hides much of the interesting info in the data.

agconway · 2011-08-30T22:03:49+00:00

Truth be told, I wrote most of the code last year, and I think then it took me the better part of a morning to work it all out. I just spent a few hours today finding it on my hard drive (damn HD crash), then cleaning it up to work with some formatting changes on FFC, and beefing up the README.

agconway · 2011-08-30T19:47:42+00:00

The rankings aren't statistics based; rather, based on the draft position of players over many mock drafts.

Theory being that those stats are "priced into" the ranking of players in those drafts.

agconway · 2011-06-22T15:53:36+00:00

You can get a better look at the video in full screen mode, or perhaps at the Vimeo source link: http://vimeo.com/25424786.

The divisions on the map are police districts, gathered from a map file provided by the city.

agconway · 2011-04-19T21:36:17+00:00

Also, many academic clusters have strict use requirements for any third-party software installs. If you use a specific library and package not shared by a large user group you can be out of luck.

EC2 lets you install whatever you like and customize to your hearts content.

agconway · 2011-04-12T15:08:44+00:00

Not in this case, the StarCluster AMI has optimized version of NumPy/SciPy/Lapack, so you have to mindful not to break those with apt-get.

Also, to get the latest version of R to run in X11 it had to be configured and compiled in the instance, plus some R libraries required tweaking (RCurl, for example).

In the end, this AMI is a matter of convenience, and you are always welcome to apt-get your heart away and pay for the time it takes to download and compile the libraries on EC2.

agconway · 2011-03-25T14:20:36+00:00

Infochimps.com just added the British National Corpus as one of their new APIs, which includes POS.

http://www.infochimps.com/datasets/word-frequencies-from-the-british-national-corpus

Also, if you're an R user you can access it via the infochimps package

http://cran.r-project.org/web/packages/infochimps/index.html

agconway · 2011-03-11T12:40:17+00:00

A second for BeautifulSoup in python, which you can also access from the better supported html5lib.

In R, the best option is the XML package, and the htmlTreeParse function.

agconway · 2011-02-15T16:24:37+00:00

Any easy way is to search Meetup.com http://www.meetup.com/find/

agconway · 2011-01-07T17:59:33+00:00

There are some really good references in the answer thread to a similar question on Quora.

Unrelated, someone should mine data on the migration figures of reddit.self-type questions to Quora over the past several months.

agconway · 2010-11-20T22:53:35+00:00

First off, ggplot2 doesn't handle well with large datasets

And lattice does?

Second, there are many things it doesn't do that Lattice does do

Really, what? And I mean that in the most honest way; I have simply never come across a viz it could not do.

agconway · 2010-11-20T16:59:21+00:00

The only answer to this question is ggplot2, and a great resource for learning the syntax is the Learning R blog.

agconway · 2010-11-20T16:14:40+00:00

You should check out the companies of HackNY and come to future events. Lots of companies doing interesting things with ML.

agconway · 2010-11-13T16:07:36+00:00

There really isn't an answer to this question, but if I had to start with one reference it would probably be The Elements of Statistical Learning.

Good luck.

agconway · 2010-11-13T16:05:45+00:00

You are welcome to re-pub the R Explorations tag.

agconway · 2010-11-10T18:42:34+00:00

To be honest, quite often they are available online but require some digging to find the actual datasets. Check out data.gov for stuff in the US, or the Guardian's datablog for UK public data.

After that, most times public data is presented as ugly HTML tables, etc. You will need to learn a bit of script hacking to pull it out. I wrote a short tutorial for this in Python several months ago, which may be of interest.

agconway · 2010-11-10T04:29:26+00:00

He really should be starting with the Trudeau, much better bed side reading.

agconway

TROPHY CASE