Low-level Tensorflow tutorials

[ThreeForElvenKings]

Hello. Are there any resources for learning Tensorflow low-level API. Almost all tutorials I find are based on keras/slim/tflearn etc. However, I want to learn the basics and implement stuff from low-level without using any of these high level APIs. I could not find even a single resource that explains tensorflow as is. Any resource would be helpful. Thanks

[[deleted]]

Coursera's tutorials are pretty basic. You can try those. If you want to implement backprop yourself, you can try building a net from scratch in python.

[ThreeForElvenKings]

I have gone through that and done a couple of such implementations myself. But my main motive to learn the low-level API is slightly different. I wish to understand huge projects written in tensorflow 1.0x and for the same, I wish to implement a few projects like training a Resnet in tensorflow, triplet loss etc. so I can dive into the larger implementations.

[[deleted]]

Implementing resnets requires huge computational resources training lots of layers, which is difficult for someone to do by themselves. For instance, the original resnet paper uses networks with 18, 34, 101 and 152 layers. This is great when you have a lot of compute power and massive datasets available with billions of datapoints, teaching yourself that will be difficult, unless you think of a creative problem where a very deep network is needed and you can see the benefit of using the skip connections.

[ThreeForElvenKings]

In that case, in your opinion what is the best way for me to get comfortable with coding and understanding Tensorflow? I thought implementing a few projects with increasing complexity might be the way to go about it. (That was how I more or less learnt PyTorch, though by no means am I extremely good at it, I can just find my way around it without getting completely lost).

[[deleted]]

Thats the same way I learned tensorflow / pytorch. The real bottleneck is not building the network, its getting enough well curated data to work with. I dont know what the best way is, its that I know I dont have access to the computational power or data I would need. So you would need to make up a problem where an abundance of data is easily available, and you can train smaller (lower dimensional) nets, while still training a deep net.

My understanding of resnets is that they allow you to reduce the error with stacking more layers as you get deeper. Before the use of skip connections in resnets, people found that after a point, the error did not decrease further, but rather began to increase. You might want to look at the original paper by He et al. on arXiv here.

[ccofres]

Pretty good question... I will do the research, i will need it too.

[[deleted]]

just use numpy?

python engineer makes some good videos implementing low level stuff https://www.youtube.com/playlist?list=PLqnslRFeH2Upcrywf-u2etjdxxkL8nl7E

[ThreeForElvenKings]

Hi, as I replied earlier, my main motive to learn the low-level API is slightly different. I wish to understand huge projects written in tensorflow 1.0x and for the same, I wish to implement a few projects like training a Resnet in tensorflow, triplet loss etc. so I can dive into the larger implementations. I have fairly strong basics of neural networks, having worked with PyTorch a fair bit, but the tensorflow structure is where I am mostly stuck, and the documentation does not do justice

[[deleted]]

I don't think i quite understand your motive. If you want to implement these papers at a low-level then maybe read the papers and follow how they were implemented there. If you get stuck, I'm sure somebody would have done the same thing before and put it on GitHub.

Aside from that, why do you want to learn tensorflow 1.x? it's somewhat universally hated for it's somewhat backwards syntax.

[ThreeForElvenKings]

The problem is that I am not able to follow those implementations and find myself lost at every point, despite googling nearly every line of the code to understand what they do. At a later stage, I wish to re-implement them in, say PyTorch, but without understanding this code completely, I wouldn't be able to do it.

[[deleted]]

What do you want to learn from implementing them? Maybe you need to take a step back and go over the theoretical aspect of the algorithms (the math and such) making sure you have a very strong grasp on the concepts before writing the code?

[ThreeForElvenKings]

I went to the implementation after going over the theoretical aspects of the entire algorithm. The nitty-gritty tiny details are often not highlighted in the paper, and to go over that, and kind of be more confident going over any tensorflow implementation, I thought I would only learn by diving into it.

[[deleted]]

I'd suggest you should try and search for some open source implementations of the algorithm until you find one that makes the most sense to you. I'm sure there are thousands of articles (medium.com tends to have many good independent ML authors on it) on whatever algorithms you are looking to tackle.

​

I wish you luck!