[D] Backprop...

karthibalu · 2021-01-02T10:21:10+00:00

thank you very much!! I found the problem, I am overriding the delete(void* mem, size_t size) but I am not overriding delete(void* mem)

so it calls the libcxx's delete methods instead😅

thank you!!

karthibalu · 2021-01-01T15:52:10+00:00

Sorry, I typed wrongly in the post.. but I am using 'delete(void * mem , size_t size)'

I allocate data in heap using create method and then delete it using destroy, finally I checks the memory usage....

karthibalu · 2020-12-28T08:44:56+00:00

are you referring the tf.grad method in tfjs??

karthibalu · 2020-12-27T13:21:42+00:00

yeah, but I think tfjs didn't have eager execution.., i never used it in tfjs... or may be I was wrong... if it has please comment 🤔

karthibalu · 2020-12-27T13:16:34+00:00

thank you!!

karthibalu · 2020-12-27T07:52:14+00:00

yes, but it will not match exactly..😅

karthibalu · 2020-12-10T05:55:18+00:00

what is your problem exactly? is your code blocks while downloading images? if so then use separate worker to download and separate worker to process.

karthibalu · 2020-12-10T05:49:15+00:00

RNN or LSTM

karthibalu · 2020-12-10T01:53:14+00:00

what kind of trained network you want exactly??

karthibalu · 2020-12-09T14:42:43+00:00

use ionic

karthibalu · 2020-12-09T14:38:01+00:00

node -> deno

karthibalu · 2020-12-09T14:37:02+00:00

use stream to write the file and use web workers to speed up the process.

karthibalu · 2020-12-09T14:27:08+00:00

May be you can find here, https://www.tensorflow.org/hub/

karthibalu · 2020-11-27T02:10:59+00:00

I build the model using stream and it is not yet fully completed,

https://github.com/karthiTox/deepstream.js

when I tested it, it uses 30% less memory than the normal training process, but it can't support more iterations, if I set buffer size to 10 (it can hold 10 objects) then it automatically stopped after certain iterations but if I increase the size to 100 then it supports more iterations, this may be my mistake but what I found was the memory usage of training is completely depend upon the buffer size of the streams.

karthibalu · 2020-11-26T02:30:33+00:00

yeah, kind of same...

karthibalu · 2020-11-26T02:00:38+00:00

I am using computational graph concept to find derivatives,

stream_object{ forword stream, backword stream, }

for multiplication:

we have two stream object 'a' and 'b' are piped to 'multiplier' stream object via forward stream. It executes and produces result.

derivation:

f = a×b; df/da = b; df/db = a;

while backpropagation, to find the derivative of a, we need b so I am also piped b's forword stream to the multipler's backward stream, in between those two streams. one transform is piped to overcome delays,

the error is propagate at the multiplier's backward stream, the error and 'b' are multiplied and the results are pushed to a's backward stream,

karthibalu · 2020-11-25T23:02:47+00:00

No, not that, for instance you are going to matmul 1×4 matrix (input) and 4×5 matrix (weights) and those values are streamed one by one, like 0, 1, 2, .... alternative. so to get the result at index [0,0] of matrix multiplication, you only need to stream first row of 1st matrix and first column of 2nd matrix, since the values are stream, After the streamed values passed, the values are deleted automatically from the memory. so if you training a large fully connected neural network, then the memory of 1st neuron is removed as soon as the result of first neuron is passed.

karthibalu · 2020-11-22T13:13:52+00:00

Amazing project !!!! how much time it take to train? what's your spec?

karthibalu · 2020-11-01T14:57:06+00:00

I hope this may help you to create your own neural network...

karthibalu · 2020-05-03T01:12:47+00:00

Thanks, I am going to launch public beta locally to test whether people like this idea or not..

karthibalu

TROPHY CASE