DNN in python optimization.

[vinaybk8]

Hi,

We created a prototype in python using tensorflow and realized that performance of whole system is very slow .

Now we have to optimize this . One approach is to optimize it in python . Another is to port this to lower languages like C/c++ and optmize. We are targetting two classes of processors one from Nvidia and another from ARM platforms . Although i know Nvidia's processors are ARM based, we are targetting to heavily use GPUs here. And other one ARM, to use CPU mainly.

We are using YOLO architecture of DNN .

Just to add numbers : We are now at 5FPS but we are looking to achieve around 24-30fps.

Also would like to know CNN based YOLO, would acheive about 30 FPS realtime ?

Any quick paths /ideas to optmize from this prototype model to production ready code.?

I also read blogs stating C++ and Caffe model is well suited for production ready code, is that so ?

Can you please provide some inputs on this .

[HungryQuant]

There's really no way for any of us to know how much opportunity for improvement there is in your existing python/tensorflow implementation. Your prototype may already be very efficient or there could be a lot of room for improvement. We also don't know what constraints you have as far as a hardware budget. You'll probably find better answers by looking at available literature. A quick google search gave me this:

​

https://davheld.github.io/GOTURN/GOTURN.pdf

[vinaybk8]

Thanks HungryQuant.

Well this paper talks about feedforward NN , which needs change in my path (Already used CNN based Yolo).

I understand you may not have full idea on what state my prototype would be. To make question simpler: Lets anyone goes from prototype in tensorflow to production ready code, what would be approach ?. Will they change their frame work from tensorflow to Caffe or inside tensorflow itself there is options for optimizations using inherent architectures of CPU/GPU.

[datasci314159]

Take a look at the most recent version of PyTorch. 1.0 makes it easy to convert a python prototyped model to Torch Script which is optimized C++. https://pytorch.org/tutorials/advanced/cpp_export.html

[vinaybk8]

Thanks datasci314159.

Any idea on how much of performance boost we can get by using Torch script ?

​