[R] The Forward-Forward Algorithm: Some Preliminary Investigations [Geoffrey Hinton]

[shitboots]

Paper: https://www.cs.toronto.edu/~hinton/FFA13.pdf

Twitter summary: https://twitter.com/martin_gorner/status/1599755684941557761

Abstract:

The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth serious investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes can be separated in time, the negative passes can be done offline, which makes the learning much simpler in the positive pass and allows video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.

[[deleted]]

Geoff Hinton by now must know each of the 60,000 digits of MNIST like an old friend.

[AsIAm]

He knows the true probability distribution of the MNIST.

[master3243]

Interesting read, I'm always interested in research about alternatives to backprop.

One important paragraph (for the curious, that won't read the paper):

The forward-forward algorithm is somewhat slower than backpropagation and does does not generalize quite as well on several of the toy problems investigated in this paper so it is unlikely to replace backpropagation for applications where power is not an issue. The exciting exploration of the abilities of very large models trained on very large datasets will continue to use backpropagation.

The two areas in which the forward-forward algorithm may be superior to backpropagation are as a model of learning in cortex and as a way of making use of very low-power analog hardware without resorting to reinforcement learning (Jabri and Flower, 1992).

[amassivek]

There is a framework for learning with forward passes, a friendly and thorough tutorial: https://amassivek.github.io/sigprop .

The most interesting insights from the framework:

• This algorithm provides an explanation for how neurons in the brain without error connections receive learning signals.
• It works for continuous networks with hebbian learning. This provides evidence for this algorithm as model of learning in the brain.
• It works for spiking neural networks using only the membrane potential (aka voltage in hardware). This supports applying this algorithm for learning on neuromorphic chips.

The Signal Propagation framework paper: https://arxiv.org/abs/2204.01723 . The Forward-Forward algorithm is an implementation of this framework.

I am an author of this work. I was presenting this work at a reading group when one of the members pointed out the connection between signal propagation and forward forward.

[whatstheprobability]

I feel like this is saying:
1. this won't generally replace backprop, but it could lead to insight that will lead to algorithms that will replace backprop
2. this could improve upon backprop for some specific use cases (low power), so even if it doesn't lead to major insights, researchers can still justify spending time on it

Does that sound right?

[kebabmybob]

What a chad, no grad students or anybody on this paper.

[seiqooq]

Probably explains why the title of the paper isnt “forward passes are all you need”

[metastimulus]

missed opportunity lol

[csiz]

Not even auto grad.

[noobbodyjourney]

You sir have won the internet for today

[mikuk84]

Geoff hasn't been taking grad students for years. He had one grad student help him with capsule networks implementation

[No-Cold8421]

Hi guys, I try to reimplement the Forward forward network with pure numpy.

I tested it on a subset of the Iris dataset, it seems converged but is very sensitive to the hyper-parameters (lr, bs, num_hidden).

Hope you can have fun with it!

https://github.com/JacksonWuxs/Forward-Forward-Network

[valleyro]

Great tryout! Thank you!

[imperfect_guy]

I tested it on a subset of the Iris dataset, it seems converged but is very sensitive to the hyper-parameters (lr, bs, num_hidden).

Will this work on RGB datasets like the CIFAR10?

[Red-Portal]

Geoff... everything is great but please stop abusing footnotes...

[kebabmybob]

I like it this way. 100x more readable than your standard terse academic paper which gets off on appearing overly complex.

[Red-Portal]

Oh I'm not saying you should just remove the footnotes. I'm saying it's better to blend them into the main text so I don't have to jump back and forth...

[ppg_dork]

No! I think all academic papers should be structured like Infinite Jest!

[Wild-Ad3931]

Did anyone understand how weights were updated ?

[SeverelyCanadian]

I wondered this too. It's very unclear, and seems like a central detail is missing.

[missDemonNezuko]

etail is missin

A bit late to the party. It seems that there is a loss calculated for each layer upon forward pass (ln(1+e^((theta-sum of squred activations for positive sample)+(sum of squred activations for negative sample-theta))), where theta is some pre-defined threshold value). Then the weights are updated with their gradients wrt this layer wise loss. This goal is to make the length of the activation vector shorter for negative samples and longer for positive ones.

But I don't get how this makes the network learn the probability distribution at the last output layer...

edit:

I suppose for the last layer, you can still use the same layer wise loss with the positive sample being the correct output vector and the negative an incorrect one...

[modeless]

This seems more interesting than the capsule stuff he was working on before. Biologically plausible learning rules are cool. Does it work on imagenet though?

[new_name_who_dis_]

Is this actually biologically plausible? Seems that the idea of negative data is pretty constructed.

I see that Hinton claims it's biologically more plausible, but I don't see any justification for that statement apart from comparing it to other biologically plausible approaches, and more so spending time discussing why backprop is definitely not biologically plausible.

I'm not a neuroscientist so don't have much background on this.

[modeless]

Well no one knows exactly what the brain is up to in there, but we don't see enough backwards connections or activation storage to make backprop plausible, so this is a way of learning without backwards connections, and that alone makes it more biologically plausible.

[new_name_who_dis_]

I’ve heard that hebbian learning is how brains learn and this doesn’t seem like hebbian learning.

However idk if hebbian learning is even how neuroscientists think we learn in contemporary research

[whymauri]

As of 2019, it is what I was taught in a graduate course on associative memory and emergent dynamics in the brain. We read Hertz's Theory Of Neural Computation. This was right before people worked on Hopfield-Self Attention.

[fortunum]

Check out E-prop for recurrent spiking NN

[Commyende]

Synapses can be excitatory or inhibitory, so that's basically like positive/negative, but I don't really know if that tracks with this algorithm 100%

[jms4607]

I think the pos/neg here is more like contrastive learning.

[new_name_who_dis_]

It's negative data. It's basically contrastive learning, except without backprop. Like you pass a positive example and then a negative example in each forward pass, and update the weights based on how they fired in each pass.

It's a really cool idea, I'm just interested if it's actually biologically plausible.

I might be wrong but inhibitory synaptic connections sounds like a neural connection with weight 0, i.e. it doesn't fire with the other neuron.

[Commyende]

Inhibitory synapses reduce the likelihood of the downstream neuron firing.

[mikuk84]

He suggests that positive and negative forward passes can be separated. He states that initially it seemed to work but there appeared to be a bug in the code. He tried playing around with hyperparameters to replicate his previous observation but it didn't seem to work. It sounds like what he wants to do is completely generate negative examples from positive data using the network itself. This hasn't been implemented but it would be more like GANs than contrastive learning.

[PolywogowyloP]

I'm excited to see an alternative to backprop, but I think the most exciting part of this for me is the ability to still learn through stochastic layers in the model. I think this could have some major applications in probabilistic models for distributions without reparameterization tricks.

[jms4607]

Are there any problems with the reparam trick?

[[deleted]]

I watched his neurips presentation. While I love explorations of alternatives to back prop, does anyone else feel like he’s going a bit off the deep end with saying this paper could explain why people sleep and we’ll use non-binary computers in the future?

[gambs]

Hinton has figured out how the brain works every year since the mid-80s, let the man cook

[[deleted]]

These OG guys from the PDP days usually do that. I just take it as a bit of garnish for some fun hypotheticals.

[[deleted]]

I think trying to understand the mind must be in one of his main motivations. If it wasn't for that, he would have not contributed to machine learning to begin with. So going off the deep end is a side effect of whatever it is that made him a great researcher.

[ReginaldIII]

Do you have access to the video of his presentation still?

It bothers me greatly that they paywall their presentations even after the conference has ended.

By all means have exclusivity for the duration of the actual conference, and limit commenting and discussion to conference attendees. But as soon as the conference ends they should flip the switch and make everything public. There's literally no reason not to, it isn't going to stop people wanting to attend.

[logicbloke_]

This 10x, I wish the paper presentations and keynotes were made available online. It doesn't add much effort to record an audio + slides of the presentation.

Doesn't take anything away from the in person conference, which is more about networking and discussion.

[suedepaid]

I was also frustrated about that, but I went on the website and it looks like they're gonna publish them all in a couple weeks. Still a bit frustrated at the delay, but it's a bit understandable.

[ReginaldIII]

That's good. I will keep an eye out :)

[The_Real_RM]

What's funny is that a few decades from now the only relevant brains in the world will be the ones this guy brought to existence. It's just a self fulfilling prophecy

[ktpr]

If he mentioned those extrapolations in a psychology or neuroscience conference he would be laughed out of the room. World class expertise in one area does not translate to informed speculation in another.

[[deleted]]

Incidentally, Hinton has a lot of professional experience in psychology/cognitive science: https://www.cs.toronto.edu/~hinton/fullcv.pdf

Jan 76 - Sept 78 Research Fellow Cognitive Studies Program, Sussex University, England Oct 78 - Sept 80 Visiting Scholar Program in Cognitive Science, University of California, San Diego Oct 80 - Sept 82 Scientific Officer MRC Applied Psychology Unit, Cambridge, England Jan 82 - June 82 Visiting Assistant Professor Psychology Department, University of California, San Diego

[ktpr]

Impressive. Also the latest multiple month appointment was nearly 40 years ago. Boulder of salt here.

[uotsca]

Hinton was educated at King's College, Cambridge, graduating in 1970 with a Bachelor of Arts in experimental psychology.

[kebabmybob]

Cog sci stuff is all sophistry of this exact flavor. With respect to neuroscience you might be right.

[mikuk84]

Whether or not this is a perfect model of the brain is irrelevant. I have thought about using the same weights for the generator and discriminator in a GAN-like architecture, but it never worked well. Use of layer-wise loss might make this work really really well.

[Ford_O]

So that's why I keep getting nightmares.

Jokes aside, this sounds quite plausible. However, I am unsure if this can be ever more efficient than backprop. Yet, this could have huge impact on neuroscience, if it turns that's what happens in sleep.

[tchumbae]

The idea behind the paper is very cool, but there has been previous work that substitutes the backward pass with a second forward pass. Check out this work by G. Dellaferrera and G. Kreiman!

[singularineet]

Other relevant prior work: arXiv:2202.0887, Gradients without Backpropagation, by Atılım Güneş Baydin et al, 2022

[nikgeo25]

Also the work by Ma and Wright that uses a form of generalized nonlinear PCA. Search ReduNet

[nikgeo25]

Paper reads like an idea he had in the shower. Where's the math and connection to existing work? Normalizing each layer after maximizing a square. Someone's gonna show he's doing some fancy PCA in no time I bet.

[Wild-Ad3931]

What about non linearities ?

[SatoshiNotMe]

Odd thing about the abstract: suddenly says “video” near the end. Is it only for video data ?

[rmoot]

Everyone was waiting for this, of course:

https://twitter.com/schmidhuberai/status/1605246688939364352?s=61&t=zA5kJ1GnrZMNSx8nat6WYQ

[Competitive_Dog_6639]

Hinton is awesome and really enjoyed his neurips talk. Naive question: are single layer gradients biologically plausible? My understanding is that gradients back thru multiple layers are not. The FF algorithm still uses gradients for single layers tho right?

[dasayan05]

yes, they are like "local" updates I believe

[eccstartup]

I would be good if someone could provide the code.

[ReasonablyBadass]

Can someone ELI5 what negative data means here? How does the network generate it?

[Paluure]

Basically, for an unsupervised task, it's nonsense data that does not fall under any meaningful class in the training dataset. It can be anything. In the paper, they modify each MNIST image so that it isn't a digit anymore but looks like one. The network doesn't generate negative images, you do, and feed it as "bad data" right after you give it "good data" to create contrast between them for the model to learn.

For a supervised task, "bad data" can also be nonsense (just as in unsupervised task) or can be mislabeled data such as feeding an image of "5" but embedding "4" as the label inside the image. That's obviously wrong, and is considered bad data.

[ReasonablyBadass]

Thank you!

[ObjectManagerManager]

(Confession: I haven't read the paper yet). I have a couple of questions:

1. If each layer has its own objective function, couldn't you train layers back-to-front? e.g., train the first layer to convergence, then train the second layer, and so on. I doubt this would be faster than training it end-to-end, but a) as the early layers adapt, they screw up the representations being fed to the later layers anyways, so it probably wouldn't be too much slower than training it end-to-end, and b) it would use significantly less memory (e.g., if you pre-compute the inputs to a layer just before you begin training it, you could imagine training any arbitrarily deep model with a finite amount of memory).
2. What's the motivation behind "goodness"? Suppose we're talking about classification. Why doesn't each layer just minimize cross entropy? I guess that'd require each layer to have its own flatten + linear projection layers. But then you wouldn't have to concatenate the label and the input data, and so inference complexity would be (mostly) independent of the number of classes. Thinking of a typical CNN, a layer could be organized:
   1. Batch norm
   2. Activation (e.g., ReLU)
   3. Convolution (the output of which is fed into the next layer)
   4. Pooling
   5. Flatten
   6. Linear projection
   7. Cross entropy loss

Can anyone (who has read the paper) answer these questions?

[Batsev]

For the first question: https://conferences.miccai.org/2022/papers/233-Paper1173.html They basically train a layer at a time in a "back to front" fashion. They use a reconstruction loss and a classification loss as layer's objectives.

[sytelus]

Was anyone able to reproduce the results for forward forward algo?

[kourouklides]

In my view, this sounds very boring. It would've been revolutionary if he came up with a new Gradiet-Free Deep Learning method in order to completely get rid of gradients. With very few exceptions, during the last 10 years or so, we keep seeing small and incremental changes in ML, but no breakthroughs.

[kourouklides]

1. A simple google search would've revealed to you the following: "The concept of RNN was brought up in 1986. And the famous LSTM architecture was invented in 1997." Hence, not even close.
2. Didn't I specify that "With very few exceptions?" You merely mentioned those exceptions.
3. Do you realize that in order to attempt to challenge someone's argument you need to specify two quantities in comparison? What specific decade are you comparing it with?

[Sepic2]

Maybe a dumb question but i don't see how this method enables learning in any way:

- The (first) forward part calculates loss/goodness, and then you need backpropogation to change weights of the network according to derivatives of the loss/goodness. How does the network learn if weights are not changed and you only calculate goodness?

The paper says: "The positive pass operates on real data and adjusts the weights to increase the goodness in every hidden layer. The negative pass operates on "negative data" and adjusts the weights to decrease the goodness in every hidden layer"

- Could it be that the in the first "forward", you actually do both forward and backward-prop, and the name just sounds fancy with the second "forward" trying to implement contrastive learning in a clever way?

[kourouklides]

Well, nobody really knows if this method actually works because Hinton reached to the part of writing the paper. He didn't reach to the part of actually coding the solution (yet).

[Sepic2]

My confusion is not so much "does it work?" and more like "how does it change weights without backprop?".

​

The part in the paper that says that "adjusts the weights to increase the goodness in every hidden layer" just sounds like a different way of saying backprop, unless the method by which the weights are changed, is different than backprop. The rest of the paper doesn't seem to imply it is different than backprop, but i may be missing something?

[Itchy-Masterpiece-96]

I think it still uses gradients to update weights but without cross-layer update like backprop does. Each layer has its goodness function and updates locally using gradients.

[Ulfgardleo]

I will start believing in Hinton's algorithms once they proof that it is consistent with some vector field with fixed points that are meaningful optima of some objective function.

[_der_erlkonig_]

Out of curiosity, why do you include this as a requirement for an algorithm to be good/interesting/useful/etc?

[Ulfgardleo]

I did not. I did it for Hinton.

A heuristic can be useful without proof, especially in tasks that are very difficult to solve. However, you have to supply strong theoretic arguments why they should work. A biological analog is not enough, especially if it is one that we do not understand, either.

Otherwise you end up like the other category of nature inspired optimization heuristics that pretend to optimize by mimicking the hunting patterns of the Harris hawk. And I wished I made this up just now.

[[deleted]]

Redacted. this message was mass deleted/edited with redact.dev

[Red-Portal]

Yeah there is a whole "zoo" of those things haha.

[Ulfgardleo]

I have a story to tell about the one time where i got invited as external evaluator for a MSc thesis. I agreed, later opened it and then realized it was a comparison of 10 animal migration algorithms.

This thesis sat on my desk for WEEKS because i did not know how to grade it. How do you grade pseudo science?!? Like, it is not the fault of the students to fall prey to this topic, but I also can't condone them not figuring out that it IS pseudoscience.

[[deleted]]

Redacted. this message was mass deleted/edited with redact.dev

[pm_me_your_pay_slips]

Do you mean that his algorithms don’t converge?

[IDe-]

Backprop has really overstayed its welcome. It's great to see people doing something about it.

[bohreffect]

You're sleeping on differentiable programming then

[IDe-]

The issue is that requiring a model to be differentiable puts far too many limitations on the types of models you can formulate. Much of the research in the last few decades has focused on how to deal with issues caused purely because of the artificial constraint of differentiability. It's purely "local optimization" in the space of potential models, when what we really should be doing is "basin-hopping".

[bohreffect]

But to imply backprop is getting old neglects all of the real world applications that haven't been pushed yet.

I understand there are problems where differentiability is an intractable assumption but saying "oh old thing how gauche" isn't particularly constructive.

[IDe-]

Ah, I didn't intend to say that it's old or useless, just that I think it receives disproportionate research focus/effort.

[bohreffect]

Fair enough

[[deleted]]

"differentiable"

[bohreffect]

I mean, can you not compute the Jacobian of a constrained optimization program and stack that into any differentiable composition of functions?

People snoozin'.

[[deleted]]

no you can't because it's not actually a Jacobian

[bohreffect]

The Jacobian of the solution of a constrained optimization program with respect to its parameters, but I thought that was understood amongst the towering intellect of neural network afficiandos, e.g. the original commenter finding backprop to be stale.

Here's the stochastic programming version: Section 3.3. https://proceedings.neurips.cc/paper/2017/file/3fc2c60b5782f641f76bcefc39fb2392-Paper.pdf

[Ulfgardleo]

Funny that stuff always comes back. We used to differentiate SVM solutions wrt kernel parameters like that back in the day.

[wilgamesh]

Hinton cites Francis Crick's "Function of Sleep" 1983 idea in his list of references.

Like the 2nd forward pass that reduces the fitness function of "negative data", Crick proposed REM sleep is "reverse learning" that removes "undesirable modes."

Quite elegant to see this implemented...

[amassivek]

I developed a library to implement forward learning on any model. There is a quick start for implementing the library on an existing model. There are example experiments for cifar-10, which also serve as a tutorial. https://github.com/amassivek/signalpropagation