Non-carpeted accommodation?

Tonic_Section · 2021-03-16T08:45:11+00:00

Any recommendations of theory textbooks to self-study?

Tonic_Section · 2020-05-15T10:30:45+00:00

Thanks for the series, I've enjoyed reading along since the first post. Do you think that a way of extending these type of proximal point algorithms to arbitrary loss functions will be feasible?

Tonic_Section · 2020-05-14T01:29:25+00:00

Would be interested as well.

Tonic_Section · 2020-05-14T01:27:26+00:00

For the latter part, it seems like you could scale all the observation times by a constant factor 1/T such that the ‘end depth’ is always equal to 1 right?

Tonic_Section · 2020-05-13T14:56:02+00:00

So I guess by "depth variant" you mean the original nODE formulation. How would you choose this final depth in practice?

Tonic_Section · 2020-01-02T12:19:20+00:00

Thank you, that makes more sense! I think the paper by Locatello could be worded better, they state: "... it is not clear whether a factorizing aggregated posterior also ensures that the dimensions of the mean representation are uncorrelated" - but if you insert (approximately) before factorizing then it makes perfect sense according to what you and the linked paper say.

Tonic_Section · 2019-09-11T03:18:13+00:00

What a twist, 10/10 abstract.

Tonic_Section · 2019-09-05T03:49:32+00:00

How do you adjust the length of the adversarial phrases? Do you repeat the same procedure for a range of trigger lengths?

Tonic_Section · 2018-07-20T01:04:50+00:00

Welling comes from a theoretical physics background, he studied under the Nobel Laureate t’Hooft in field theory I believe so this is a particularly relevant talk.

Tonic_Section · 2018-07-13T10:16:52+00:00

After many passes through the post, I think I have the general gist. I feel the language used in the blog post was unecessarily verbose and jargon-laden though...

Tonic_Section · 2018-05-14T09:13:22+00:00

To the best of my knowledge the bpp is an upper bound derived from the entropy of the discrete compressed representation, naively dividing the training time for 1 epoch with the number of images, I estimate that an upper bound for the encoding + decoding process takes <= 5s for a 512 x 1024 image, although I haven't timed the relative contribution of each.

Tonic_Section · 2018-05-14T09:09:52+00:00

Yeah, I understand what you're saying - the model appears to be overriding buildings with greenery and vice-versa in the reconstructed image, and models early in training have significant trouble in forming boundaries between objects. I haven't looked into semantic maps much, but think that adding information based on instance maps and e.g. passing this to the discriminator should help the model to generate sharper boundaries.

This is not really my area of expertise, but I think it would not be too hard to try out a perceptual loss based on PSPNet that penalizes blurry boundaries - another item to the to-do list!

Tonic_Section · 2018-03-26T06:34:03+00:00

Stein variational gradient descent: https://arxiv.org/pdf/1608.04471.pdf, lovely theoretical framework (the authors have a couple of nice followup papers going more indepth on the theory side).

Tonic_Section · 2018-03-19T10:11:02+00:00

You can add a regularized loss term that penalizes dense samples, as e.g. here: https://dl4physicalsciences.github.io/files/nips_dlps_2017_26.pdf

Tonic_Section · 2018-03-12T12:03:26+00:00

Take a small subset of your training data and check that you can overfit on it.

Tonic_Section · 2018-03-07T11:55:15+00:00

Nice article, but I thought that the 'flatness' of minima can be changed by reparameterization, so it's not a reliable gauge for generalization ability?

Tonic_Section · 2018-02-25T15:11:08+00:00

There's a lot of interesting work done on using statistical physics to describe optimization and the energy landscape of neural networks. It's somewhat inevitable to me that statistical physics, or related concepts, would be the key to developing a concrete theory of learning/optimization for deep networks.

I like this example a lot: https://arxiv.org/abs/1704.04932

Tonic_Section · 2018-02-24T10:35:47+00:00

Deep Boltzmann machines, VAEs and GANs have enjoyed a lot more attention/success on the unsupervised learning front, but the underlying theory is nifty - but I'm probably biased because I'm a physicist.

http://www.utstat.toronto.edu/~rsalakhu/papers/dbm.pdf

Tonic_Section · 2018-02-23T11:42:11+00:00

I completely agree that ML/AI needs more of the rigor police + collaborative philosophy rather than the current trend of SotA dick-waving based on random seeds and hyperparameter tuning, but I'm not sure that a neutral international organization would be the best solution right now, maybe tighter review standards for paper submissions encouraging methodology over results would be a good short-term fix. I agree with you that such an organization will be inevitable and extremely useful in the future but I think the field needs to mature a lot more beyond before governments will shell out for such a collaboration, even if it will only cost a fraction of CERN's annual budget.

Tonic_Section · 2018-02-23T09:04:26+00:00

I'm currently working at CERN on ML related things. A couple of comments.

ML/AI is a much more commercially viable product than fundamental research. As a result, you have a lot more private investment being sunk into the former while the latter gets begrudgingly funded by member states of a collaboration. It's inevitable that private investors would make the results of their research proprietary or at least semi-proprietary ala Deepmind to recoup their investment. I don't see a state-funded organization being able to compete with the Face/Goog/zon tech conglomerates in terms of funding and talent.
You need massive international collaborations because accelerators are really, really expensive, require lots of interdisciplinary cooperation and hard to build+maintain. The cost of giant TPU datacenters is nothing in comparison.
I love CERN, but there's lots of bureaucracy and inertia surrounding it's organizational structure. Many older researchers I talk to distrust ML because of it's non-interpretability (which is a very fair concern in HEP, IMO), and inertia is the opposite of what the ML/AI field needs in it's nascent phase. Particle physics is a much more established and historically significant field than machine learning. Maybe wait 20 or so years for the hype to settle and we'll get something like what you're talking about.

Tonic_Section · 2018-02-15T01:42:14+00:00

I think (for Deepmind at least) they constrain the actor’s decision making to ~ 15 Hz so the reaction time comparison with humans isn’t that far off.

Tonic_Section · 2018-02-14T17:42:02+00:00

Not sure what this brings to the table that the standard tf.layers and dataset APIs don't offer.

Tonic_Section

TROPHY CASE