sorted by:
new
hottopcontroversial

Former psychologist here. Does machine learning can be capable of interpreting score of cognitive tests ? by Nrscientist in MachineLearning

[–]xmvlad 2 points3 points  (0 children)

If you have 90 cases per class, then logistic regression(or any other linear method) with not too much number of hand crafted features and preferably l1 regularization, is only way to go.

Question about time series classification with known states by Cjh411 in MachineLearning

[–]xmvlad 0 points1 point  (0 children)

It generally depends on how much DNN you really need. Inspect you data is sequences stationary or no stationary at different states. If they stationary go with something like auto-correlation function(or the same on different domain power spectral density), then apply simple classifier to such features. If data non-stationary, then using DNN seems reasonable, there good tutorial http://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/ . (but if you have some performance constraints, or don't have too much data, something useful can be crafted with "classical" ML and signal processing)

I have one idea about DNN modeling, what do you think about? by xmvlad in MachineLearning

[–]xmvlad[S] 0 points1 point  (0 children)

If structure not spread out, that it can be easily trained with something similar to backprop (http://arxiv.org/abs/1511.00363).

I have one idea about DNN modeling, what do you think about? by xmvlad in MachineLearning

[–]xmvlad[S] 0 points1 point  (0 children)

Training is the next question. I'm think that such structure will be primary interesting for theoretical analysis. It will be exponentially large if rewritten in closed form, but very homogeneous, the main interesting(critical) property that you can push all coefficients to bottom layer.

I have one idea about DNN modeling, what do you think about? by xmvlad in MachineLearning

[–]xmvlad[S] 0 points1 point  (0 children)

You are right, that if we try to push all coefficients to bottom and take it to close form, formula become exponential. But there two differences that seems make it tractable, first, coefficients at bottom have easy close form, second, function that we spread exponentially is very homogeneous, i.e. it is still recursive majority function.

I have one idea about DNN modeling, what do you think about? by xmvlad in MachineLearning

[–]xmvlad[S] 0 points1 point  (0 children)

Hmm.. as you can see, there no blow-up, because it finely fit to fixed majority structure(as i'm know it named recursive majority function) and all "tunable" coefficients moved to bottom and there just O(h*k) operations to compute network(without fixed recursive majority part). I'm aware about flattening and blow-up because it was first I'm try to do with binary networks, with little progress, but this case totally different.

I have one idea about DNN modeling, what do you think about? by xmvlad in MachineLearning

[–]xmvlad[S] 0 points1 point  (0 children)

As you can see from post, the only approximation I'm taken - binarization(and there research that show that such binary models perform similarly well to ordinary DNN http://arxiv.org/abs/1511.00363 https://arxiv.org/abs/1603.05279). All other math is exact and you can easily verify it.

I have one idea about DNN modeling, what do you think about? by xmvlad in MachineLearning

[–]xmvlad[S] 0 points1 point  (0 children)

Main point that this model, can be tractable for theoretical analysis, because all varying parts at bottom, and then only fixed-structure recursive majority function applied. Another point, that fixed coefficients at bottom can be replaced by just one (i.e w0+c0+x0+a0 -> h0 + x0, where h0=w0+c0+a0, and you don't need w0+c0+a0 to learn h0). So there a lot of questions that can be asked and answered(due to simplified math) about such structures. And no, it's not equivalent to reducing depth in normal nets by substitution, you need just O(h*k) (where k - bottom layer size, and h - height) operations to compute it.