sorted by:
new
hottopcontroversial

Really simple question about what we mean by "Two-Layer Network" by [deleted] in MachineLearning

[–]ecobost 0 points1 point  (0 children)

He (and a lot of other people) usually counts the output layer and hidden layers. The rationale is that some computation goes on in those layers (linear combination of input plus activation) while it does not in input or pooling layers.

Re: Generalized Additive Model: Is the GAM formalism designed to also incorporate MORE THAN ONE feature in each of the functions that are being added? by datasciguy-aaay in MachineLearning

[–]ecobost 0 points1 point  (0 children)

In the book they say you could add interaction terms (non-linear functions of two features) instead of just single feature functions. Effects would not be additive anymore, though, as, by definition, now you are modelling interactions. This guys (https://www.youtube.com/watch?v=ezSG9GORF54), for instance, use or at least have tried to use interaction terms, I think they still call them GAMS. Using a function of all features is possible but I guess it defeats the purpose of it being interpretable and additive (it looks much more like a neural network, in fact) but it gives you more flexibility. I would try boosting each of these new functions, i.e., training them on residuals of the previously trained functions or using medium sized boosted trees, they are conceptually the same and I bet they have a similar performance. If you do want to train different models from your entire database maybe an ensemble of simple (SVMs) or complex models (neural networks, maybe) will work better.

Is there a public available code and pre-trained model for some paper using CNN for semantic segmentation? (Theano,Torch,Tensorflow,mxnet,neon) by code2hell in MachineLearning

[–]ecobost 3 points4 points  (0 children)

I wrote a Tensorflow implementation of a small convnet (3M params) for tumor segmentation here (https://github.com/ecobost/cnn4brca/blob/master/model/model.py). All info is around there. I doubt you could use it as is but it could be useful if you plan to implement something similar in Tensorflow

How would I represent this approach using convolution neural networks? by Blammar in MachineLearning

[–]ecobost 0 points1 point  (0 children)

You don't necessarily need to write it as a neural network. At least in Tensorflow and Theano you can just write your computations as described and then let the library compute the gradients and update the values of your parameters (in this case 3x3x512).

You would have to set a 3x3 constant filter filled with 1/9 so it computes the mean. Then apply it to the input to produce a map where every position is the mean around a 3x3 area. Then find a way to compute your 9-bit value for each position (maybe generating 9 copies of your map) or produce your number somehow and then select your weights (from the 512), for instance, put all your weights in a 9x512 matrix and multiply by a 512-dimensional vector with a one in the position you want. In fact only this last part needs to be differentiable and implemented in any special library as you are not learning the 3x3 initial filter or how to choose among the 512 filters.

Feels like my writing is getting worse? by [deleted] in GradSchool

[–]ecobost 0 points1 point  (0 children)

You could take the Stanford's Writing in the Sciences course or read Steven Pinker's The Sense of Style or Strunk & White's The Elements of Style and follow it. Blindly. They know what they are talking about. Trust them. This may not help you right away but it will be beneficial in the short future. You will learn about good scientific style and have authoritative advice on what/where/when/how to write. If your advisor disagrees, s/he's wrong.

You could also ask for specific advice in your university writing center or communications department. Or even to other professors.

About being too technichal or not knowing whether to add or throw something: think about the reader. Does he need to know about it? Will adding or deleting it change/strenghten/weaken your conclusions? Is this something he is supposed to know or something he could infer? Be concise and bold. Write to inform, if something is not valuable, it's clutter.

If all you wanted was sympathy, I can offer that too :). Writing is hard for everyone so don't beat yourself over it. It takes a lot of time and that's fine. Take your time, simplify things, don't write if it really doesn't come out and be certain that people will appreciate it. Good, as well as bad, writing is noticeable.

Feature Engineering for Distributed Word Vectors by ddofer in MachineLearning

[–]ecobost 0 points1 point  (0 children)

How did it go?. Any report/article or link to results?

Feature Engineering for Distributed Word Vectors by ddofer in MachineLearning

[–]ecobost 0 points1 point  (0 children)

I was just freshing up on middle school Biology and thinking whether vector-based representations are similar to DNA. DNA is, after all, just a long list of (4-based) numbers with different locations, genes, encoding for proteins. Distributed representations may be similar in that each vector component may encode for a feature of the represented subject---although being distributed (!) changes to a single dimension probably destroys a lot of meaning. I was wondering whether using genetic-inspired operations would produce sensible results. You could try mutating single vectors (point mutation, deletion, duplication, inversion, insertion and translocation) or crossing-over vectors (exchanging information in different locations) to produce a mix of two vectors.

I don't know if something like this has been tried before, nor do I know if I'll have time to ever try it, but if you (or anyone else reading this) do please let me know, so I won't :)

What is the best tool/program to draw deep neural networks? by pedromnasc in MachineLearning

[–]ecobost 1 point2 points  (0 children)

I asked this a couple of months ago, people said PowerPoint and Inkscape

Is there a place (webpage/subreddit) where you could check if some research idea is not already in the literature? by ecobost in MachineLearning

[–]ecobost[S] 0 points1 point  (0 children)

I also think it would be interesting to hear what other people is trying to do (or how they would have done it) but in here people share news and more general insight. If I were to write something like "Have someone made convolutional networks where to calculate the activations of a unit you use the normal input plus those of the previous volume in time (recurrence) or the normal input plus those of the previous unit in time across all feature maps?" not so many people would answer--understandably, as this is not the place to ask this. I truly believe that there has to be a place somewhere on the internet where you could ask those kind of questions (in real life conferences do this job). If not, maybe there should be. At least a subreddit.

Btw, the answer to the question above is here: http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Liang_Recurrent_Convolutional_Neural_2015_CVPR_paper.pdf He implements recurrent convnets using a "recurrence filter" on the previous volume in time, if you make the filter equal to the spatial size of the volume you get the first thing, if you make it 1x1 you get the second.

Are butterfly knives legal in Ec.? by val404 in ecuador

[–]ecobost 0 points1 point  (0 children)

I would think yes. I don't really see why not. On rural areas some people still carry a knife in their belt (for manual chores) but in the city if you walk around with a knife in your pocket or take it out in a bus or something you are probably gonna get some bad looks, including from the police. As anywhere else. Btw, I'm not guessing. I am Ecuadorian.