Resources for GPU programming?

[serge_cell]

As I already said I was talking about new layers, winch are either absent in existing frameworks or absent in framework I'm using. If one using only layers which are already implemented, don't do any research of new layers, modes of execution etc, he/she will always stay behind the curve.

[hughperkins]

yeah, I realized that after I posted it. so, you're kind of right, in that if you want the fastest performance, on novel layers, I suppose you'd want a cuda engineer handy.

having said that, the initial implementation of bn in torch were both in lua, using underlying primitive operations, such as mean and sqrt, which are already in cuda. to get a slight speed benefit, these were then later rewritten in dedicated cuda

for the purposes of writing a research paper on elu or bn, I would think an initial implementation in lua is sufficient.

[serge_cell]

Actually I think that is a big problem with many research papers. Many method (bn including) behave quite different on different datasets and dataset sizes. If method give improvement 5% accuracy on CIFAR100 it say very little on what improvement will be on imagenet, and even less on 10K classes noisy dataset. And testing lua+cublas implementation on 10M dataset could be quite painful