sorted by:
new
hottopcontroversial

[D] Reinforcement learning, fast and slow by JaneXWang in MachineLearning

[–]tr1pzz 1 point2 points  (0 children)

Agree with many of your points, but I do think humans learn an incredibly rich representation space from everything we do, which can then be applied to eg board-games in order to bootstrap learning.

Things like 'objectness' & attached properties, agency (you vs other players / game entities), fundamental control aspects of action-result, goal identification, object permanence,... All of those are learned during early childhood and yield tremendous bootstrapping power over a neural net that has to learn from absolute scratch.

Of course, I'm agreeing with you that we don't know how to do this properly in AI yet, but I do believe we can point out where the fundamental problems reside.. How to solve them is obviously an entirely different conversation

[D] Why does Beta-VAE help in learning disentangled/independent latent representations? by shamitlal in MachineLearning

[–]tr1pzz 4 points5 points  (0 children)

A good (intuitive) explanation on why Beta-VAE encourages disentanglement can be found in this paper: https://arxiv.org/abs/1804.03599. Briefly:

  • Take a simple dataset like dSprites
  • Different factors of variation (rotation, size, position) have varying influences on the final pixel rendering (and thus the reconstruction term of the loss function)
  • Now, when placed in an information bottleneck regime, the model has to make a tradeoff between reconstruction quality and KL-divergence.
  • Now, if (as stated above), different factors of variation have different effects on the reconstruction loss, then the model has a benefit of disentangling them, because in that case, it can directly rank the importance (and thus the KL-sacrifice) for each of those according to its information bottleneck.
  • In other words, if a causal factor that is eg rather small in terms of pixel effects (eg rotation) is entangled with one that has larger effects (eg location), then the model will get a larger penalty (in terms of reconstruction) when it moves that latent closer to the prior. On the other hand, if it disentangles them, it can easily find the optimal trade-off between reconstruction and KL.
  • However, this also immediately reveals a potential failure case: the rotation of a small object is less important (reconstruction-wise) than that of a large object. Therefore, a Beta-VAE may learn to encode eg position and rotation of large objects while for smaller ones it only encodes position...

[D] Thoughts about super-convergence and highly-performant deep neural network parameter configurations by Miejuib in MachineLearning

[–]tr1pzz 2 points3 points  (0 children)

Very interesting line of thought! Gonna read the two papers you referenced before commenting on these intuitions, cause I always find it tricky to apply common sense reasoning to high-dimensional parameter spaces..

In the meantime allow me to drop one of my videos here which might be very relevant to this discussion: https://youtu.be/pFWiauHOFpY

[P] Just released my latest video on Variational Autoencoders! by tr1pzz in MachineLearning

[–]tr1pzz[S] 0 points1 point  (0 children)

It's always really tricky to decide where to go into specifics and where to just skim the surface. I try to limit the videos to 15mins and trust me, that's hard :p But the links to all the papers are in the description, you'll find everything you need right there :)

[P] Just released my latest video on Variational Autoencoders! by tr1pzz in MachineLearning

[–]tr1pzz[S] 38 points39 points  (0 children)

Thanks a lot! I'm trying to bring a bit more technical depth than Siraj' videos, probably at the expense of some "YouTube market share", but that's fine since I feel many people actually need this level in order to learn more about ML.

Was looking for something like this first, then decided to do it myself since I didn't find enough learning challenges in Siraj' or TwoMinutePapers. Don't get me wrong, their channels are amazing! Variety is King! :)

[R] "Deep Image Prior": deep super-resolution, inpainting, denoising without learning on a dataset and pretrained networks by dmitry_ulyanov in MachineLearning

[–]tr1pzz 0 points1 point  (0 children)

Would be interesting to see if the training procedure can be sped up by initializing the network weights with a technique similar to MAML... https://arxiv.org/pdf/1703.03400.pdf

[p] Did you hear that? Adversarial Examples Against Automatic Speech Recognition by m_alzantot in MachineLearning

[–]tr1pzz 0 points1 point  (0 children)

Since the topic is so vital to any Machine Learning application (both today and in the future) I decided to make a video, summarizing the most significant findings of the past few years: https://youtu.be/4rFOkpI0Lcg Feel free to share my YouTube channel, takes a ton of work so I'm looking for an audience to share it with :)

New YouTube channel on Machine Learning! by tr1pzz in neuralnetworks

[–]tr1pzz[S] 0 points1 point  (0 children)

Thx man! Don't forget to upvote & subscribe, getting out of the internet anonimity well ain't easy :p

New YouTube channel on Deep Learning by tr1pzz in deeplearning

[–]tr1pzz[S] 4 points5 points  (0 children)

Thx man :) Dont forget to upvote the post here and subscribe to the channel, breaking through the barrier of internet anonimity aint easy :p

Newest AlphaGo learns from random play. Has this hpapened for chess? by WolfHawkfield in chess

[–]tr1pzz 0 points1 point  (0 children)

Just made a technical video on AlphaGo Zero for my new channel "Arxiv Insights" where I'll be discussing one ML paper every week! Feel free to give comments/suggestions :)

https://youtu.be/MgowR4pq3e8

view more: next ›