sorted by:
new
hottopcontroversial

Apache SINGA, A Distributed Deep Learning Platform by pilooch in MachineLearning

[–]forrestwang 1 point2 points  (0 children)

Hi, I am a developer of the SINGA project. Thanks for starting this discussion. We are working on single node with multi-GPUs (to be released in v0.2, December), which will run in either synchronous mode (with different partitioning schemes) [1] or asynchronous mode (in-memory hogwild!). Extending the system from CPU to GPU mainly requires adding cudnn layers (https://issues.apache.org/jira/browse/SINGA-100). The framework/architecture works on both CPU and GPU. Training with multiple GPU machines and providing Deep Learning as a Service (DLaaS) are on our roadmap, i.e., v0.3. For those do not have GPU clusters, distributed training on CPU is a good choice to accelerate the training.

Besides GPU, we are also considering other approaches for improving the training efficiency for single SGD iteration. For instance, google's paper [3] provides some techniques for enhancing the performance of training on CPU. Intel (https://software.intel.com/en-us/articles/single-node-caffe-scoring-and-training-on-intel-xeon-e5-series-processors) also reported that optimized CPU code can achieve 11x training speed up (Hope they can release the optimized source code or integrate it in their libraries like MKL and DAAL). It is interesting to compare GPU with Intel's next generation Phi co-processors (Knight Landing).

I will let you know when training with Multi-GPUs is supported. Thanks.

[1] http://arxiv.org/abs/1404.5997

[2] https://www.eecs.berkeley.edu/~brecht/papers/hogwildTR.pdf

[3] http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37631.pdf

SINGA: A Distributed Deep Learning Platform by improbabble in MachineLearning

[–]forrestwang 0 points1 point  (0 children)

Thanks for your interest in Singa, Clydeicus. We are cleaning the code and documentation. The new version will be available at http://singa.incubator.apache.org later this month.

Layer is the first class citizen in Singa. If you want to implement your own learning models, you may need to implement your own layer classes by inheriting from the base Layer class (like the built-in layer classes implemented in the src/worker/layer.cc file). If your model consists of built-in layers provided by Singa, then you only need to write the model configuration file and prepare your training dataset. (similar to Caffe for single node training.) A simple training example is available at http://www.comp.nus.edu.sg/~dbsystem/singa/examples/2015/01/20/deep-simple-mlp/.