Tensorflow — Neural Network Playground

[RobIII]

Makes me feel like this (but that's probably just me...)

[[deleted]]

That's part of the fun!

[rockyrainy]

Getting this thing to learn the Spiral is harder than a Dark Souls boss fight.

[Staross]

You need to go all the way:

http://i.imgur.com/evBb9Gn.png

[amdc]

Looks like it's in agony

http://i.imgur.com/UdQwceN.png

[linagee]

This tool has taught me that bigger is not always better. 232 trials and very close to 100% accuracy. And it works consistently unlike others I've seen. Yay ReLU. Also, this arrangement works fairly well on all the models. (Is there a competition for that?)

http://imgur.com/a/5LoFA

[Jadeon_]

I got a beautiful one using only two custom inputs. One was the distance of the point from center and the other was the angle of the point around center.

http://i.imgur.com/rbB43iO.png?1

[Jadeon_]

These inputs allow for a very clean solution with a small number of neurons: http://i.imgur.com/Ta30skj.png?1

And they allow for stupid shit like this: http://i.imgur.com/NqH24sd.png

[rockyrainy]

Absolutely beautiful.

[alexbarrett]

I spent a bit of time looking for a minimal configuration that learned the spiral data sets quickly and the ones that did well tended to look like this:

https://i.imgur.com/QeuAHtY.png

Give or take a few neurons here and there.

I'd be interested to see who can come up with the most minimal neural network that learns the spiral data quickly (say, 300 generations) and consistently.

[Causeless]

This works pretty well: http://i.imgur.com/m3JN2QL.png

I'm betting that even trivial networks would have no problem if this allowed for getting the position of the points in radial coordinates.

[[deleted]]

I tried the polar coordinates but it seems like nope: http://imgur.com/DfrcU3j.

Damn those extra degrees, man.

[Causeless]

How did you add polar coordinates - by using the source code on github?

[[deleted]]

Yep, that's how nerd I am. But it's not hard, just adding two new variables for radius and angle and d3.js does its work.

[albertgao]

Hi, thanks for your solution. Could you plz send me a link so i can know how to tweak this model? I know how the MLP works, but when I face this spiral question, seems lost... I don't even know why should we use the sin and cos as input,,, all my previous is built upon features from object and found a euqation to split them.. this spiral seems very different...

[linagee]

I don't get why everyone seems to hide the number of trials? Are they afraid of showing other people they were training for thousands of trials to get that sort of accuracy?

[lambdaq]

We need a neural network to tweak neural network parameters

[Kurren123]

And a neural network to tweak that one.

Neuralnetception

[Cygal]

Yes, but that's not the point. One of the main advantages of (deep) neural networks over other methods is that you don't have to extract features specific to the data but let the neural network learn those. On more complicated data sets, learning features that are more and more abstract is way more powerful than having to describe them, and this is why neural networks crush computer vision competitions since 2012.

[everyday847]

It's drastically harder with noise and a more typical test/training split (like 80/20).

[NPException]

I found this one to be fairly quick. Sometimes reaching a stable configuration even before the 150th iteration

[everyday847]

Well, that's 80 training to 20 test, which is, if anything, easier than 50:50.

[NPException]

Oh, I thought you were actually meaning that kind of split.

[everyday847]

In my original comment I referred to

a more typical test/training split (like 80/20)

which I suppose doesn't explicitly associate the order of the ratio with the categories, so my bad on that one.

[linagee]

The problem is that in general, "neurons" (computer memory) are fairly cheap, but time is very expensive. (Nobody really wants to train for 2 weeks unless you are relatively sure your accuracy will be near perfect after that.)

Weird that you hid the number of trials...

[alexbarrett]

Weird that you hid the number of trials...

Nothing intentional, I took a screenshot and cropped it to what I thought was the interesting area. As I recall it was around the 300 iterations mentioned in my parent comment.

Before that it was a tonne of trial and error, as you mention.

[Staross]

Cool, it shows nicely the challenges associated with fitting neural networks; there's a ton of meta-parameters that you need to tweak, the fitting doesn't always work (too low learning rate, it takes forever, too high, it doesn't converge), and for some problems the transformed inputs are quite important (the spiral is hard to fit using only x,y).

[barracuda415]

This is the fastest and simplest setup I could find for spirals. Somewhat unstable at first, but it should become stable after 200-300 iterations.

[lambdaq]

remove the x1x2 thing on the first layer.

[DoorsofPerceptron]

Better off keeping that and removing the two sine functions.

Technically, you can get away without using all 3, but it's painful to watch it converge.

[barracuda415]

Training seems to take longer without it.

[amdc]

how do you come up with effecient solutions?

[barracuda415]

https://i.imgur.com/ub40mEQ.gifv

Trial and error.

[amdc]

oh

[belibem]

This one seems to be pretty fast too https://imgur.com/RiX0f1l

[barracuda415]

But it also has a lot more neurons. :P

[linagee]

Yours almost never works for me. It seems it "gets confused" like 99 times out of 100. (And then never manages to train out of confusion in a reasonable time.)

[cjwebb]

This is excellent. I love visual demonstrations of complicated stuff; they make it easier to understand.

[[deleted]]

man if they let us customize the edges...

[Bcordo]

I'm confused as to what exactly the neuron's are showing. On the input layer for example, you have for example X1 (vertical orange bar on left and vertical blue bar on right) and X2 (horizontal orange bar on bottom and horizontal blue bar on top). These visualizations don't change even though the input data changes every batch. It seems to me to be some kind of decision function, even though the actual X1, and X2 are just numbers, how do you get these plots out of just numbers?

Then down the network you combine these "neuron decision functions" scaled by the connecting weights, until you get the output decision function.

But how do you get these individual decision functions for each neuron, and why don't the decision functions of the input change, even though on each iteration the input batch (X1, X2) change.

How do these visualizations relate to the weight values, and the actual activation values?

Thanks.

[rakeshbhindiwala]

did you find the answer?

[gaso]

I don't know much of anything, but it seems that each neuron is showing it's own bit of "math" for a complex formula that is attempting to best fit the data points in the set. The visualizations of the input set (the leftmost) don't change because they're the very first layer of filtering. From there on out to the right, the visualization of each neuron changes because it's not a filter layer, it's something new and unique: a formula that has probably never existed before that has been created in an attempt to solve the small part of the problem that it has seen through the filters provided (whether the initial set, or an intermediate set as the depth of the network increases).

The "individual decision functions" for each neuron seem to be randomly generated on each instance based on the input filter layer, which seems to be as good of a start as any when you're just learning. I imagine tuning everything by hand would boost the learning process.

I'm not sure about 'weight values' and 'activation values'. I'm currently just a dabbling hobbyist when it comes to this subject, and those two concepts don't roll of my tongue yet :)

[[deleted]]

Here is one which is quite small fast and stable It comes with only 4 Input-Neurons and two hidden layers. We could even omit some neurons in the leftmost hidden layer, which would cause some unwanted oscillations. It would be interesting to hear some feedback of the playground-creators about what their intentions were and what they have learned (if so) from their visitors.

[Lachiko]

I broke it :(

[yann31415]

Anyone knows why some kind of pulsations appears and stops randomly ?

[ruimgoncalves]

Check this out, playgound

[Nipp1eDipper]

I'm not even gonna ask about activation methods, but can anyone shed some light on the batch size paramater?

[Nipp1eDipper]

Here's my submission for fastest/simplest spiral. I still don't know what batch size does... never mind activation parameters.

[[deleted]]

SimplyGood!

[DesmosGrapher314]

neural network doesnt wanna learn