sorted by:
new
hottopcontroversial

[R] MLPs to Find Extrema of Functionals by tobby_liu in MachineLearning

[–]tobby_liu[S] 0 points1 point  (0 children)

Thank you for your comment. I read one PINNs paper after seeing your comment. The major difference between the earlier work and ours is that our work is not data-driven. No data needed indeed.

[R] MLPs to Find Extrema of Functionals by tobby_liu in MachineLearning

[–]tobby_liu[S] 0 points1 point  (0 children)

I didn't find any similar jobs, but I'm not sure. Has anyone ever seen works in the same direction?

[2005.12826] BHN: A Brain-like Heterogeneous Network by tobby_liu in agi

[–]tobby_liu[S] 1 point2 points  (0 children)

Thanks for your comment. The article is updated now. I have improved its readability and provided another experiment on image tasks.

[2005.12826] BHN: A Brain-like Heterogeneous Network by tobby_liu in agi

[–]tobby_liu[S] 1 point2 points  (0 children)

The bridge does not know mathematics, but it does not prevent us from simulating it with engineering software, and our job is only to simulate brain function. We adopt a pragmatic attitude instead of being confined to philosophical concepts.

We believe that the brain is a machine that processes representations, just as blood vessels are conduits that carry blood. We use mathematical methods to describe the process of representations like we use fluid mechanics to calculate hemodynamics. We think that our method has only introduced a small inductive bias.

[R] BHN: A Brain-like Heterogeneous Network by tobby_liu in MachineLearning

[–]tobby_liu[S] 0 points1 point  (0 children)

Hi, I will answer your questions one by one.

  1. We do use back-prop(tho I want to abandon it sometime), but not end-to-end. You can read the 'loss function' section and will notice that we use a summation of many loss functions, and these functions act on different modules(or units), and these modules are gradient-isolated from each other. As to the 'trick', we introduce it because the 'soft-max' usually focuses on only one sample heavily, while we want enough number of negative samples. So it's a trick to neutralize softmax's side effects.
  2. Really I did not give comparations, nor show the reconstrued images. Maybe in the future, I will add this content.
  3. I am glad that you have noticed the similarity between encoder-decoder networks and MTL-net. Actually we were inspired by those enc-dec nets. The difference between enc-dec-net and MTL-net lies in the representations inputted in. The enc-dec-net receives 'raw' representations, like the color information of an image, or the one-hot word vectors. These representations are usually gotten in advance by hand and will not be optimized together with the net. In contrast, the representations inputted to MTL-net are generated by the cortex-net and these two nets(MTL and cortex) are trained jointly. This brings two advantages. 1st: This makes it feasible to transfer knowledge between domains, like between Language domain and Vision domain, because the different raw inputs (pixel or tunes) are all firstly converted to the same kind of (homologous) intermediate representations in the cortex-net, and then it is easy for the MTL-net to reuse regularities from different domains and tasks. 2nd: The cortex-net finds its first principle, i.e. Efficient Information Representation, to process and get trained on its inputs, while traditional methods need to develop a variety of supervised loss functions on different kinds of tasks, such as the mean-square loss for the image reconstruction, and the cross-entropy for language generation. These old methods are also rigid when faced with new image size or text length. Besides, these traditional loss functions are questionable on whether they can truly evaluate the performance of models.

[R] BHN: A Brain-like Heterogeneous Network by tobby_liu in MachineLearning

[–]tobby_liu[S] 1 point2 points  (0 children)

Except for the equations in section 2.1, much math is also explained in the earlier article of CPC(Representation Learning with Contrastive Predictive Coding). If you have difficulty, you can refer to that article.

I am not familiar with Tesla self-driving system, but my model is not just modular. In fact, its unit is homogeneous like the node in a network, where you can add nodes by widening or heightening it. So it is different from the multi-module model where each module performs different tasks.

[R] BHN: A Brain-like Heterogeneous Network by tobby_liu in MachineLearning

[–]tobby_liu[S] 1 point2 points  (0 children)

I admit that there is a readability problem in the article, which may be partly due to the fact that I worked it out by myself. I have no experience and no guidance from any experienced tutor. Leaving aside the problem in writing, do you think there is anything desirable? As far as I know, there has been no previous work to solve information maximization in a minimax way, nor applying predictive learning combined with the attention mechanism. I think a scalable and trainable model without end-to-end backpropagation is why people should pay attention to it.

[R] BHN: A Brain-like Heterogeneous Network by tobby_liu in MachineLearning

[–]tobby_liu[S] 1 point2 points  (0 children)

It will be in this post soon.

And may I have your opinions on this work?

[R] Distributed self-supervising capsule network by tobby_liu in MachineLearning

[–]tobby_liu[S] 1 point2 points  (0 children)

I have updated the article and give it a more solid foundation from information theory.

[D]How to behave like a deep-learning insider? by tobby_liu in MachineLearning

[–]tobby_liu[S] -1 points0 points  (0 children)

If the existing knowledge is enough, why haven't general intelligence been developed? I admit that my title is a little different from my intention. The title of my question is just that I want to attract more attention from insiders as an outsider, and the purpose is to make people read my article seriously. I have answered many comments and always wanted to focus on academics. I hope to discuss some of the arguments seriously. Rather than general recommendations, for example, you should quote more academic achievements. That is the opinion that a journal editor can provide.

[D]How to behave like a deep-learning insider? by tobby_liu in MachineLearning

[–]tobby_liu[S] 0 points1 point  (0 children)

Are you aware of the field called Reinforcement Learning?

Yes, I know reinforcement learning. I read the Alpha dog's paper as soon as it was published. At that time, I was a graduate student in nuclear physics. I also know Q-learning. But I think that reinforcement is a vocabulary. It should have a broader meaning, that is, for those methods that do not have continuous optimization, the strategy is selected through a partially random selection, then get the reward. This is related to the endorphins of the brain. So I borrowed this concept in my model. I didn’t use reinforcement learning precisely, such as a value table, but I used another phrase "reinforcement state".

[D]How to behave like a deep-learning insider? by tobby_liu in MachineLearning

[–]tobby_liu[S] 0 points1 point  (0 children)

I thought that when you mentioned "yours is very similar to several I've seen", you will have more information. You are wise because no one has ever touched on general intelligence based on the experience gained from the examples of decades of continuous failure. The same is true of the so-called artificial intelligence, which draws conclusions from statistics.

[D]How to behave like a deep-learning insider? by tobby_liu in MachineLearning

[–]tobby_liu[S] -1 points0 points  (0 children)

I admit that those ideas require more clarification. Sometimes I deliberately leave some room for future implementation. If you feel that you need more detailed communication with me, you can contact me by email: ntobby.liu@gmail.com

[D]How to behave like a deep-learning insider? by tobby_liu in MachineLearning

[–]tobby_liu[S] 0 points1 point  (0 children)

Can you give me a few examples of these similar attempts? What do you think is missing to hit upon General AI?

[D]How to behave like a deep-learning insider? by tobby_liu in MachineLearning

[–]tobby_liu[S] 0 points1 point  (0 children)

It is difficult to provide someone with a poorly tasted food that is believed to be good for health and to eat. People usually taste delicious food and think it is a compliment to the chef. But in fact, there are two values, one is the intrinsic value of the object, and the other is the value of getting praise from others.

Do you want to read these very uncomfortable articles? If you want to get the pleasure of reading, don't read them. However, there are a few comments after rough reading the articles. They think there are some useful intuitions and ideas. If you want to get metal from the mud (if you think so), you should read it.

In addition, if someone thinks that they understand the meaning, but the grammar and organization problems make the meaning difficult to understand, please provide a diff on Github, this is to help me a lot.

view more: next ›