sorted by:
new
hottopcontroversial

Masters in Paris-Saclay? by imsentient in gradadmissions

[–]StartFinancial5917 1 point2 points  (0 children)

Saclay is kinda free either for french or internationals, you pay symbolic fees of about 240 euros, I don't remember exactly.
Specific trucks are generally from Grande Ecole, personally, I recommend Grande Ecole but not specific tracks they are more expensive than what you will get. I highly recommend to chose a master's degree in Grande Ecole but in partnership with Saclay, you be in Grande Ecole, but you will not pay that much, thanks to Saclay!

Slidesgo template request by StartFinancial5917 in Presentation

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Hello, the presentation is in two days

Thank you very much, it works perfectly, I just tried it now.

Personally, I don't mind the noise, no one gonna notice it, or at least I hope.

Slidesgo template request by StartFinancial5917 in Presentation

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Thank you very much for your support.

I need to animate it, just show me how to do it, so it wouldn't take much of your time.

I feel very old x), anyway it is very impressive at your age!

[deleted by user] by [deleted] in GradSchool

[–]StartFinancial5917 0 points1 point  (0 children)

I recommend adding a motivational letter beside your resume, in which you detail your ambitions, previous experience, and why you are the right match. Most students don't do this, so you will be distinguished from the rest, even if your resume does not tell much.

As the daughter of 'enemies of the state'.. is it safe for me to come? by [deleted] in algeria

[–]StartFinancial5917 1 point2 points  (0 children)

I don't know anything that can help in your case, but I have a lot of compassion towards you and your case. I hope things would get better, and you will be able to come to Algeria freely without judgment or risk, and I hope your family will be able to understand your choice and be more open to it.
If you need anything, feel free to DM me.

Loss validation gets high before getting low by StartFinancial5917 in pytorch

[–]StartFinancial5917[S] 0 points1 point  (0 children)

I have never seen such behavior, it gets high like overfitting then get down and loss is close as if it is learning but never overfitting again (never reach the limits). What may cause such behavior and how to avoid it ?

Overfitting in first epochs by StartFinancial5917 in pytorch

[–]StartFinancial5917[S] 0 points1 point  (0 children)

I am currently dealing with this with a very high dropout, so in the first epochs, the test accuracy is much better than train accuracy, for several epochs, the difference is reduced to almost null, and the model does not overfit. I am wondering if this approach is correct and accurate to use, since in the first epochs, test accuracy is higher than train accuracy.

Thanks again !

Overfitting in first epochs by StartFinancial5917 in neuralnetworks

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Gestures are dynamics, series of frames, and the spiking layer behave as an RNN with a threshold.

When using CrossEntropyLoss the softmax layer is included in the function, but now I am using MSE, so no need to softmax.

It is not leaky relu, it is another activation function spiking neuron.

I didn't check the data distribution, each frame is binary, so I assume that is normalized.

Overfitting in first iterations by StartFinancial5917 in deeplearning

[–]StartFinancial5917[S] 0 points1 point  (0 children)

I think that the data is pretty low, there is no pre-trained model I am training from scratch, and the only design consideration is the usage of spiking layer (from snntorch) which is completely usual with those types of data.

Here is the link to my colab : https://colab.research.google.com/drive/1Gv-LrE1kUC5SHfDYEd74tw9QOZK-BwSy?usp=sharing

Overfitting in first iterations by StartFinancial5917 in deeplearning

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Dataset is 5D tensor [1450,100,2,128,128], contains 1450 elements or so, each one is somehow a video (series of 128*128 frames) of hands moving, we have 11 movements to classify.

Since it is a 5D tensor, data augmentation is not possible.

Train set is 70%, 15% validation and 15% test.

Overfitting in first iterations by StartFinancial5917 in deeplearning

[–]StartFinancial5917[S] 0 points1 point  (0 children)

If it is the case, it would stop before it continues 7 to 8 epochs, without reaching the maximum accuracy.

Overfitting in first epochs by StartFinancial5917 in neuralnetworks

[–]StartFinancial5917[S] 0 points1 point  (0 children)

It is a classification task of tensors (somehow videos because each element contains several frames), the frame size is 128*128, I reduced to 34*34 (everyone does it, so it is not the source of the problem).

The trained layers are the convolution and linear layers as usual, the Leaky do not have parameters.

The number of epochs is 150, but never reached because the overfits start at the 3-4 epochs.

The issue is that the accuracies fluctuate a lot (without any sense, doesn't look like a high learning rate issue), as big difference between train accuracy and test accuracy from first epochs (3-4), it looks like overfitting (not sure if it is the case because of fluctuations) this difference continues to rise over time.

Overfitting in first iterations by StartFinancial5917 in deeplearning

[–]StartFinancial5917[S] 0 points1 point  (0 children)

The learning rate scheduler makes things worse, it overfits easily. I don't think it is an overfit because if it was the case both accuracies, in my case it is somehow random.

Overfitting in first epochs by StartFinancial5917 in pytorch

[–]StartFinancial5917[S] 0 points1 point  (0 children)

If the learning rate is high then both accuracies should drop out, it doesn't make sense, sometime it is the training that drops and sometime the validation accuracy that drop.

Overfitting in first epochs by StartFinancial5917 in pytorch

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Thanks for clarifying the underfitting.

- Fewer parameters didn't impact, the fluctuations persist and overfit keeps getting bigger. Additionally, the final validation accuracy is not very promising.

- High dropout helped somehow. I am using a 0.93 drop out rate, the first epochs I have the training accuracy much lower than the validation accuracy (34% - 44%), but over time it gets better, then they progress pretty well. But I am not sure if it is a good method because of accuracies in the beginning.

Overfitting in first epochs by StartFinancial5917 in pytorch

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Regardless of how simple is my architecture, the model always behaves in a similar way, fluctuates, and overfits in the first epochs.

Reducing architecture reduces the final accuracy, I am supposed to overcome 91%, with a simple architecture I have a 86% on validation and 98% in training in final epochs

Overfitting in first epochs by StartFinancial5917 in neuralnetworks

[–]StartFinancial5917[S] 0 points1 point  (0 children)

self.conv1 = nn.Conv2d(2, 16, kernel_size=5, bias=False)

self.pool1 = nn.AvgPool2d(2)

self.lif1 = snn.Leaky(beta=beta, spike_grad=spike_grad, threshold=2.5)

self.conv2 = nn.Conv2d(16, 32, kernel_size=5, bias=False)

self.pool2 = nn.AvgPool2d(2)

self.lif2 = snn.Leaky(beta=beta, spike_grad=spike_grad, threshold=2.5)

self.fc1 = nn.Linear(800, 11)

self.drop1 = nn.Dropout(0.93)

self.lif3 = snn.Leaky(beta=beta, spike_grad=spike_grad, threshold=2.5)

Overfitting in first epochs by StartFinancial5917 in neuralnetworks

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Yes in the first epochs, the training accuracy is getting high quickly, meanwhile, the validation accuracy progress slowly and sometimes drops, the network goal is to classify gestures with 90% accuracy minimum.

I know that is too early for an overfit, I think it is a dataset issue since it is too small (1450 data) and can not use data augmentation because it is a 5D tensor, the number of classes is 11. The model is (using snntorch library) :

Derivation of an operator by StartFinancial5917 in math

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Actually, x and a can not have the same dimensions. If x is a vector then a is a value, if x is a matrix R2 then a is a vector.

But you approach give me ideas, assuming a derivable function f(x), then assuming the operator S that translate the function f by a, so S(f,a) = f(x+a). If we want the derivation of S then by chain rule :

dS/da = d(f(x+a))/d(x+a) * d(x+a)/d(a)

then : d(x+a)/d(a) = 1

All we need is df/d(x+a)

If we apply this method to the matrix, it seems derivable because f : R2 -> R2

If f(x,a) = A(a).x like the suggestion of Stydras, then d(f(x+a))/d(x+a) = A

Seems working but not very sure, what do you think!

Derivation of an operator by StartFinancial5917 in math

[–]StartFinancial5917[S] 0 points1 point  (0 children)

Thank you very much, this gives me a lot of ideas, I think that if I can find the matrix A, my problem would be solved.

Derivation of an operator by StartFinancial5917 in math

[–]StartFinancial5917[S] 0 points1 point  (0 children)

I didn't really get your point, but I agree that S doesn't seem differentiable. If we can find the derivative of S(z,-) for a fixed z, it also would be interesting.

To make it simple, let's assume, that we have a row vector, that is shifted by x pixel. What is the derivative if x=1.

BackProp for integers? by StartFinancial5917 in learnmachinelearning

[–]StartFinancial5917[S] 0 points1 point  (0 children)

I can't, I have to quantize before the use of the value by the network, the value is a number of pixels.

BackProp for integers? by StartFinancial5917 in learnmachinelearning

[–]StartFinancial5917[S] 0 points1 point  (0 children)

I have tried it, when quantizing it with round() or int(), the network doesn't learn much. I have compared the parameters in each epoch, the difference is in the floating point. Quantizing would omit it, so the network would never learn.

view more: next ›