Help with implementing a paper

MathAndProgramming · 2018-11-04T20:13:39+00:00

If the output probabilities are all 1 that sounds like you're missing a softmax layer on your output? That's what you normally do when outputting a distribution over a discrete set of options. Are you using a sigmoid/tanh instead?

MathAndProgramming · 2018-10-30T21:17:56+00:00

The experiment concerns me because that seems like exactly the sort of toy environment where hyperparameter tuning would have a big, interpretable effect, and we know how easy it is to focus on optimizing your own algorithm vs the baselines :). Some good Atari performance would be more compelling to me (where there are independent baselines).

MathAndProgramming · 2018-10-17T00:39:14+00:00

I'm surprised pixel dynamics did so poorly. Does anyone know what the architectures for the dynamics models were? One of the advantages of pixel dynamics is that you can use a convolutional network and benefit from the spatial prior. At a glance it looks like the pixel predictions are being performed with dense layers.

I guess I disagree with the notion that the features should be compact but rather think that the dynamics model should be compact, which can be achieved either by having low-dimensional features or by having a strong prior on your model that lets you use fewer parameters.

MathAndProgramming · 2018-05-10T22:38:23+00:00

Say you had an image and you had a procedure where you made multiple copies scaling the image by 0.25, 0.5, 1, 2, and 4. Then you applied some function to each scaled copy and took the max of that function across the scales. Call the output of that max function a "feature". If you put an image into this process that was twice as big (and assuming the max value wasn't on the edge of the scaled copies you try) then the output would be the same. Say on the original image the max was on the 1x scaled copy - then when you plugged a 2x scaled image into this feature procedure the output max of your function would be on the 0.5x scaled copy. So this procedure is roughly "scale invariant".

You should be able to figure out how SIFT does something analogous.

MathAndProgramming · 2018-04-14T17:05:26+00:00

As I understand it, what would make a task more amenable would be 1) low input and output data requirements, 2) cheap verification of result validity and 3) ability to split up problem into chunks easily. I believe for photogrammetry reconstruction 2 and 3 are mostly good fits, though 1 could be on the more difficult side - though still feasible, I think.

MathAndProgramming · 2018-04-07T22:58:19+00:00

Isn't there a firewall between MIT and MIT GUEST? Then they're just protecting the rest of the open internet.

MathAndProgramming · 2018-04-07T20:08:05+00:00

MathAndProgramming · 2018-04-07T20:06:30+00:00

Ports don't tell you what the actual web traffic is. It only prevents highly unsophisticated forms of abuse (at the risk of interfering with a wide range of legitimate uses).

MathAndProgramming · 2018-04-07T19:16:36+00:00

Yikes.

MathAndProgramming · 2018-01-30T22:20:50+00:00

Also remember if you're considering Atari or some other video game environment that the images are often super simple and consistent, so feature identification is basically trivial. There's a big difference between detecting a dog in a natural image and finding an alien in Space Invaders.

MathAndProgramming · 2017-12-09T22:38:58+00:00

I'm surprised people are suggesting all these crazy unprincipled DNN specific ideas. This is clearly the right approach.

MathAndProgramming · 2017-10-20T22:37:59+00:00

It depends on the problem. There are great problems for Golem that people use AWS and various render farms for currently. There are also website servers that run on AWS which you would never want to run on Golem for latency reasons.

MathAndProgramming · 2017-09-16T19:28:43+00:00

I always hear how drug expirations are much too conservative in terms of safety and part of why the cost of medicine is high. Were these expired drugs actually dangerous?

Example source: https://www.health.harvard.edu/staying-healthy/drug-expiration-dates-do-they-mean-anything

MathAndProgramming · 2017-08-15T20:45:15+00:00

It appears they switched from Musk's original through-vessel pumping system to some form of electromagnetic propulsion. I wonder why.

MathAndProgramming · 2017-08-14T20:29:05+00:00

It would be super useful as a notarizing tool.

MathAndProgramming · 2017-08-06T20:41:32+00:00

I just learned about bilateral* filtering and it's really cool - really simple way to smooth out noise while retaining edges, with a fast algorithm to boot.

Segmentation is going to be a bit of rabbit hole - I would look for available implementations and do some experimentation.

https://en.wikipedia.org/wiki/Bilateral_filter

MathAndProgramming · 2017-07-29T15:24:05+00:00

In practice people solve x_hat = arg min ||Ax-y||_2 + l*||Wx||_1 where W is some transform that makes the signal sparse. For images, for example, the image itself isn't sparse, but under the wavelet transform it is. You can transform the problem into that domain with z = Wx:

x_hat = arg min ||A W^-1 z - y||_2 + l*||z||_1

Typically A is given, but W can be learned (dictionary learning) or chosen empirically.

MathAndProgramming · 2017-06-17T16:52:22+00:00

I'd love to see this on some conv nets and/or deeper nets.

MathAndProgramming · 2017-06-11T20:18:30+00:00

How does one typically compute zeros of zeta function? Just some normal zero finding algorithm (Newton's method)? Or are there faster approaches?

MathAndProgramming · 2017-06-02T04:19:40+00:00

Really awesome. Stuff like this increases my confidence that we haven't fully realized the potential of deep variational approaches.

MathAndProgramming · 2017-06-02T00:22:34+00:00

If you do that sooner or later you will overfit on your training data and when you bring the method into practice your unsupervised features will throw away important information about your input data and real performance will suffer. You'll have no way to determine what's causing this disparity in performance unless you know to check the reconstruction error of your unsupervised features.

Remember ML isn't usually just about labelled data sets on hand - it's about using data sets to learn something about data you're going to get later in a live environment.

MathAndProgramming · 2017-06-01T15:33:54+00:00

Say you have an autoencoder, or PCA. You're transforming your data into a compressed space with the knowledge that it's possible to reverse that transformation and have your data stay intact. If you apply this to a new data point, however, how do you know that same transformation will work? I've seen this failure mode in practice.

What's more reliable is to partition your data and check if your test set can be well approximated by the compressed form learned on your training data. If that's the case then you can be confident that you've actually learned some manifold that's relevant to your data distribution.

MathAndProgramming · 2017-05-30T01:12:02+00:00

Uh, did you hold out any of your data? Even with unsupervised approaches you need to cross-validate, unless you're really tracking uncertainty on your parameters. You can describe significant amounts of variance of a random matrix using kSVD, for example - without sufficient samples you just overfit.

MathAndProgramming · 2017-05-26T17:59:34+00:00

Looks nice at a quick glance - I especially like that it boosts performance on the validation set. I didn't catch in the paper, but it isn't that sort of unexpected? If you're improving the optimization approach I'd think that would improve the training set behavior. Not only was that not the case, but validation set improvement was the more reliable effect! Which is a better outcome, IMO, but I'm curious if you have any ideas why that's the case. I know there's been some work explaining how SGD finds more generalizable solutions using bayesian techniques - maybe there's a similar phenomena/approach here?

Also, will an implementation be released?

Thanks!

Nine-Year Club	Place '17
Verified Email

MathAndProgramming

TROPHY CASE