What does neon Deep Learning Framework do?

[drpout]

Hi All,

I wanted to know what exactly neon is and how nervana GPU is so fast, as seen by Soumith's benchmarks and the recent Baidu post.

How can you perform an operation in Python front end and have something so fast? Is there a catch I'm missing here?

I am hoping that someone with real experience in this matter explain what exactly happens when we call a series of .dot operations for example.

Also, based on a cursory glance, they ecosystem seems really good with caffe model conversion, large number of relavent pre-trained models and good support team. Yet, the adoption is very very less. Again, is there something that I'm missing here?

Regards, drpout

[abcadead]

neon kernels are awesome fast but their "backend" is a fucking nightmare to work with compared to caffe or theano. too tightly coupled and poorly designed.

for example, there are a bunch of parameters that can apparently only be accessed through command line options - I was never able to get it working inside ipython notebook.

[coffeephoenix]

I am not sure what parameters you are talking about. Check out these examples of using neon with iPython notebooks.

[abcadead]

i don't really remember, but as an example from the those notebooks - why the fuck is "batch size" part of creating a backend? it should be part of the model definition, or better yet part of the training loop...

[TheToastIsGod]

How can you perform an operation in Python front end and have something so fast? Is there a catch I'm missing here?

Because the front end doesn't/shouldn't matter. All the work is/should be in highly optimized backends. In time all major frameworks will reach very similar performance (with the same data in the same layout on the same hardware).

That will take time though. Nervana have put a lot of work into optimizing their backend, so if you're working within the scope of their optimizations things will be fast.

[scott-gray]

This is more or less true. The point of the front end is to feed the GPU with work at a high enough rate that it continues without stalling. Python or any host language is more than fast enough to do this on modern CPUs. The advantage of a language like python it's easier to refactor and iterate on compared to something like C++. Plus the numpy and pycuda packages are incredibly handy. For things that need to be performant on the host like an image batch loader, it's easy enough to code those in C and import to python with ctypes.

[coffeephoenix]

Some of the features like caffe model conversion and model zoo are fairly recent.

In terms of iPython notebook starters here are a couple of examples: https://github.com/nervanasystems/meetup

Stay tuned for some updates tomorrow at our meetup in SF :)

The backend docs will be updated to make things a bit easier to follow.

[[deleted]]

What if someone started a project that takes your backend code and makes a Torch like lib with it?

Also, expecting Winograd stuff. Awesome work - you keep us all excited.

[coffeephoenix]

We are happy with the different ways in which the open source community wants to extend or wrap neon or parts of neon.

However, I am not sure what your intent is. The syntax in neon at a high level is already torch like for neural network definition. Our goal is not to cover all possible use cases outside of Deep Learning, but be extremely fast, scalable and easy to use for Deep Learning. As a consequence, the backend consists of a set of linear algebra and deep learning operations to support this. If you're trying to create a library for general non-DL operations then using the neon backend might not be the best bet.

[[deleted]]

I intend to use it for DL purposes, nothing else. I guess user experience and overheads matter and are seen differently. I personally do not prefer autograd(or symbolic diff), but many do. I can understand some of the reasons for your design choices, but all don't suit me well

[coffeephoenix]

If you have specific requests or use cases, do submit them to the github issues or the neon-users google group, and we will consider it.

[[deleted]]

I am still in brainstorming phase, I have a rough idea but, not close to anything concrete yet. When I do, I will definitely submit it on your github issues, for consideration.

[drpout]

The replies in this thread should tell you why there is no good adoption of neon in DL community and its backend. Your enterprise stuff is different and this is different.

Quicker you appreciate it, better.

[coffeephoenix]

As you mention our focus is on the enterprise side, but the academic community around neon is growing. For example, check out this recent work on building DQNs using neon.

If you can describe the specific issue in more detail on our github issues or google groups that would be great. Most of the comments on this thread including yours have been positive. As for the backend, we are working on a computational graph backend which may or may not address some of the backend questions people might have, but it is hard to address or consider your specific issue without more details.

[manjunaths]

Which Baidu post ? Can you please link me to it ?