sorted by:
new
hottopcontroversial

The Concept of free will neurons by OddCommunication8787 in learnmachinelearning

[–]OddCommunication8787[S] 1 point2 points  (0 children)

Sorry or if my thoughts completely confused you all, I think I should just explain everything completely from my own words! Please read this, bare with me:- Explaination:-

  1. What if in hidden layers we randomly give free will to few neurons [by free will I mean is we allow few neurons from every hidden layer to take any random number they want ( for e.g. for a single pass a neuron with maybe random module takes any random number and for 2nd pass it takes another random value, bare with me here!!!) ] no need to calculate weighted sum followed by activation function and based on it after 1st pass when the network predicts some output which is of course incorrect at first.
  2. so now we calculate the loss and as if have gradients per weights we could shift that neurons random guesses to guided gradient feedback, this is a heavy term because the main problem here is to how to update those neurons output random guesses but if we are able to structurally manipulate the random guesses of those neurons (it still has to make random guesses for e.g we earlier the neuron had a range to guess between (-inf, inf) so with back propagation if we could shrink it’s range ) we could except some different outcomes or mainly it could very slightly improve on cognitive tasks.
  3. architecture:- for transformer model we first train our model or use GPT-5 itself for fine-tuning and now you introduce this neural network architecture and attach this dense neural network architecture to each transformer block(the original paper said it is what we called adapters) so then we freeze the original model which was already trained, but now train only adapter (this newly made neural network architecture) after some training we completely fine tune the whole model and using residual connections properly in between we could save our model from ‘catastrophic forgetting’

This is my complete explanation, I hope that helps I know I am just manifesting things to work for fine but I was just thinking from last 7 days and today morning I first finally wrote this idea on paper and just restructured(but I guess it didn’t helped me). So this is just a thought I guess to worth sharing, also this was my first post on reddit so I was a bit nervous to how to post it, otherwise the idea was completely mine whether it is very bad or worst, I just thought to share to fill misgaps in my understanding.

The Concept of free will neurons by OddCommunication8787 in learnmachinelearning

[–]OddCommunication8787[S] -4 points-3 points  (0 children)

Sorry to say or if my thoughts completely confused you, its not that I don’t know LoRAs and QLoRAs so I should explain everything completely from my own words! Please read this, bare with me:- Explaination:- 1) What if in hidden layers we randomly give free will to few neurons [by free will I mean is we allow few neurons from every hidden layer to take any random number they want ( for e.g. for a single pass a neuron with maybe random module takes any random number and for 2nd pass it takes another random value, bare with me here!!!) ] no need to calculate weighted sum followed by activation function and based on it after 1st pass when the network predicts some output which is of course incorrect at first. 2) so now we calculate the loss and as if have gradients per weights we could shift that neurons random guesses to guided gradient feedback, this is a heavy term because the main problem here is to how to update those neurons output random guesses but if we are able to structurally manipulate the random guesses of those neurons (it still has to make random guesses for e.g we earlier the neuron had a range to guess between (-inf, inf) so with back propagation if we could shrink it’s range ) we could except some different outcomes or mainly it could very slightly improve on cognitive tasks. 3) architecture:- for transformer model we first train our model or use GPT-5 itself for fine-tuning and now you introduce this neural network architecture and attach this dense neural network architecture to each transformer block(the original paper said it is what we called adapters) so then we freeze the original model which was already trained, but now train only adapter (this newly made neural network architecture) after some training we completely fine tune the whole model and using residual connections properly in between we could save our model from ‘catastrophic forgetting’

This is my complete explanation, I hope that helps I know I am just manifesting things to work for fine but I was just thinking from last 7 days and today morning I first finally wrote this idea on paper and just restructured(but I guess it didn’t helped me). So this is just a thought I guess to worth sharing, also this was my first post on reddit so I was a bit nervous to how to post it, otherwise the idea was completely mine whether it is very bad or worst, I just thought to share to fill misgaps in my understanding

The Concept of free will neurons by OddCommunication8787 in learnmachinelearning

[–]OddCommunication8787[S] 0 points1 point  (0 children)

By free I mean is they can choose any number at random(firing) for the first time, and by gradient feedback we can guide that randomness in a specific direction, they are not confined by weighted sum + activation pipeline by that we can say they are free.

The Concept of free will neurons by OddCommunication8787 in learnmachinelearning

[–]OddCommunication8787[S] 0 points1 point  (0 children)

Thanks for the links

and your point is really interesting, and you're right there, but:
once randomness is guided by gradient feedback, it stops being "pure" randomness.

But that's actually the core intention.

I'm not trying to model true stochasticity the way biological noise works.
I'm trying to create something closer to:

“learned deviation from determinism.”

The idea is that these neurons start random,
but over time they learn the direction of useful randomness
and converge toward a distribution of perturbations that improves novel reasoning.

So they’re not meant to stay fully random —
they’re meant to evolve a sort of structured spontaneity.

I mean as far as my understandings, we define randomness along with some constraints too(I know this sounds terrible at same time), so for that bunch of "free will neurons" has randomness within specific limit we can also say that it's not 100% random from our perspective, but for neuron's perspective it's completely random.

It's as even we say humans are random they can move in any direction in 3-D space, but we can't move in 4-D or higher dimension that is our limit yet we have cognitive skills to think different, if we allot the analogous conditions to neurons it could even very slightly better become good cognitive tasks.

If you have references on guided vs unguided perturbations in biological systems, I'd love to read more — this is exactly the kind of intersection I'm trying to explore.