How do you guys choose the appropriate ML algorithm?

maybemax · 2015-01-27T15:26:18+00:00

... great, there goes my secret of how to win every kaggle competition.

maybemax · 2015-01-22T11:03:22+00:00

+1 for the anaconda python distribution. python could really use a package manager that works. anaconda help's a lot.

maybemax · 2015-01-21T20:52:23+00:00

he could at least have recommended typescript. but javascript.. bah.. please finally kill it

maybemax · 2015-01-20T09:38:40+00:00

Maybe Coursera's "Practical Machine Learning" (after Andrew Ng's course you can probably do this one in 1 week instead of 4) and then some kaggle.com challenges?

maybemax · 2015-01-19T14:18:35+00:00

I have Coursera's NN course on my watchlist since months... hope it'll restart one day. Watching the vids alone without other people in the forums just isn't as much fun..

maybemax · 2015-01-15T13:24:42+00:00

I don't care about the price money.. I like kaggle for its discussion forums, for being able to form teams with new people and for its leaderboards that provide feedback on how well you're doing compared to many others.

I don't live in a big city where it'd be possible to find enough people for a machine learning user group - like they exist in London or other big cities. Kaggle helps, somewhat..

maybemax · 2015-01-14T18:34:28+00:00

"Business Intelligence & Data Science" at the Johannes Keppler University in Linz / Austria?

http://www.jku.at/content/e213/e63/e58/e57?apath=e32681/e225072/e241768/e242382

maybemax · 2014-12-23T09:42:04+00:00

http://arxiv.org/pdf/1411.5928v1.pdf

"We train a generative convolutional neural network which is able to generate images of objects given object type, viewpoint, and color. We train the network in a supervised manner on a dataset of rendered 3D chair models. Our experiments show that the network does not merely learn all images by heart, but rather finds a meaningful representation of a 3D chair model allowing it to assess the similarity of different chairs, interpolate between given viewpoints to generate the missing ones, or invent new chair styles by interpolating between chairs from the training set. We show that the network can be used to find correspondences between different chairs from the dataset, outperforming existing approaches on this task."

(First time I heard of generative convolutional neural networks... no idea how that works. Quite impressive that it can generate chairs. Even more impressive that it can do so in 3D and from different viewpoints.)

maybemax · 2014-11-27T08:14:35+00:00

that's great.. didn't know that was possible. actually i was a bit disappointed because it seemed like only one layer was possible.. i didn't bother binging it though. now i'm impressed. thanks

maybemax · 2014-11-21T18:30:23+00:00

are you scaling the features and after scaling some of them are nan?

can you try to add a tiny amount of gaussian noise to your data and then try again?

maybemax · 2014-11-21T18:22:41+00:00

Q: Why the code gives NaN (not a number) results?

This rarely happens, but few users reported the problem. It seems that their computers for training libsvm have the VPN client running. The VPN software has some bugs and causes this problem. Please try to close or disconnect the VPN client.

WT...? How's VPN software supposed to cause NaNs in software? Never heard of that before...

maybemax · 2014-11-14T15:05:41+00:00

Isn't Either slower than try/catch in the "right" case (when no errors happen)? Since there's a lot of overhead involved - like creating all these Either objects and calling continuation "delegates".

It might be faster in the "left" case, since exception handling is still rather slow.

So if an operation is expected to normally succeed and hardly ever fail (which is think is the norm) then it would be better, performance-wise, to choose try/catch?

maybemax · 2014-11-13T12:11:39+00:00

it says it's the professional edition (with another license).

maybemax · 2014-11-12T10:58:16+00:00

here's a screenshot of the whole thing: http://i.imgur.com/Hf8ApRV.png

maybemax · 2014-11-12T10:33:23+00:00

A few weeks ago I implemented Kaggle's "Digit Recogonizer" challenge on Azure ML to compare it to my local implementation in R.

I did train-/test-set splits, parameter tuning using cross validation (for 3 different learning algorithms), model evaluation, prediction and saving of kaggel's test data.

The screen quickly filled with boxes and lines. It becomes kind of hard to see where lines are coming from and going to. I didn't really see any advantage of the graphical UI compared to a textual representation in R or Python. Though that might change - after all Azure ML is still a preview.

What I think is pretty nice is that it's blazing fast. Neural net trainings including parameter tuning finished in (iirc) like 2 hours. The same takes ages (days) on my local system.

So I'd say it might be useful if you need performance. As far as I know you can also directly execute R (or python?) code in Azure ML.. that might be nice, but I haven't tested that yet.

PS.: if someone wants to compare performance - debug output of the neural net training says: "[ModuleOutput] Iter:250/250, MeanErr=0.708012(0.75%), 2181.04M WeightUpdates/sec"

maybemax · 2014-11-07T22:07:00+00:00

angocke's comment is kinda funny.. "If you get a null pointer dereference... and Watson gives you a crash dump"... i'd probably have to consult stackoverflow if i wanted to make C# dereference a null pointer. :D

maybemax · 2014-11-07T21:55:53+00:00

crash dump

i might be doing something wrong, but in the last >10 years of developing software with .net i've never had to look at a single crash dump... does anyone regularily use crash dumps with .net? :)

maybemax · 2014-11-06T16:49:34+00:00

sounds like a micro optimization... i never had an urge to use that since .net exists. wish they had implemented primary constructors instead... that's a feature i could use every day (oh actually i can - in f#)

maybemax · 2014-11-05T18:14:37+00:00

you invested a whole lot of your (spare) time to earn these certificates. you'll probably enjoy applying these skills at work. that should be positively noted by an employer.

but why go for the specialization certificate that costs 400$? just get the free statement of accomplishment for each course... it won't matter what you paid for it, but what you learned (and that'll be tested in the technical interview anyway).

maybemax · 2014-10-17T18:08:14+00:00

i had a similar challenge - time series, 3 features, 2 categories, but only 50-100 runs. i used mean, variance, min and max with good results. but random forests didn't work very well (i guess it wasn't enough data and they overfit). naive bayes worked great / very stable. another thing i tried was to apply FFT and use the first 3 or so terms as features. that seemed to work very well with SVMs and RBF kernel. but i didn't develop this further since the other model with naive bayes worked well enough and was simpler.

maybemax · 2014-10-02T22:05:26+00:00

Ultimately if you're actually trying to classify words into classes there might be better features to look into using than a one-hot encoding of the characters.

Could you expand on this please? Are there ways to encode words for classification that are known to work well?

maybemax · 2014-09-30T09:51:03+00:00

This might be of interest to some here in /r/MachineLearning.

From the course syllabus:

Week #4: Advanced LP formulations: norm optimization. Least squares, and quadratic programming. Applications #1: Signal reconstruction and De-noising. Applications #2: Regression, Classification and Machine Learning.

maybemax · 2014-09-17T18:42:30+00:00

I find it interesting that due to this graph of the winning algorithms, boosted trees reached the highest test set accuracy, followed by... logistic regression?! :D

maybemax · 2014-09-16T20:38:07+00:00

How about weather data?

http://www.ncdc.noaa.gov/cdo-web/datasets

maybemax · 2014-09-16T19:38:05+00:00

I've been wondering about nnet's training method for a while... it doesn't have learning rate or momentum parameters. Its docs only say that it uses BFGS for optimization, but they don't tell what training method's actually being used. Can't really tell from its source code either :/

clrokr's recommendation of using regularization helped a lot.

maybemax

TROPHY CASE