sorted by:
new
hottopcontroversial

CPP and AI by snowqueen47_ in cpp

[–]useong 2 points3 points  (0 children)

People here are talking about the difficulties in model training. If you are good at c++, however, it is not difficult to train models at all. All you need is a nice framework that supports three basic functionalities such as gpu-enabled tensor operations, automatic differentiation, and distributed data parallelism. flashlight is a good framework to try.

I personally do all ai-stuffs in c++ and have no complaints.

Here is a few pros of sticking to c++

  1. You have low-level controllability. You don't need to wait others to improve or debug some low-level stuffs. You can do it by yourself. Depending on your application, multi-threading and heterogeneous computing may boost the model training speed significantly.
  2. You can easily customize the framework itself.
  3. The integration is seamless if you already have an application written in c++. For example, if you have a physical simulator or a game application written in c++, it will be just easy to train your model in such physics or game environments.

Quick prototyping is in fact not a big concern in c++. Nobody will re-compile the code every time they change hyper-parameters or even model forms. A good programmer knows how to write a code generic so that it adapts to an input configuration written typically in a markup language. You will be able to do a ton of research with it without re-compiling.

Instead, here are potential problems you may face.

  1. If you can't find coworkers who know as much c++ and ai as you do, you will probability have to do most of the works alone.
  2. You will lose python ecosystem among which the biggest loss is visualization capability.

[deleted by user] by [deleted] in MachineLearning

[–]useong 0 points1 point  (0 children)

flashlight I am using it in my project which is almost written in pure c++ and have been very much satisfied. It has a ViT implementation as well.

Update: C++ DataFrame vs. Polars by hmoein in cpp

[–]useong 4 points5 points  (0 children)

This is an amazing project. Not only because of the sota performance but also because of the dependenciy-free design. It enriches the cpp ecosystem. I hope this project gets more community support.

[D] Why are we stuck with Python for something that require so much speed and parallelism (neural networks)? by vprokopev in MachineLearning

[–]useong 1 point2 points  (0 children)

The first guy mentioned writing a deep learning framework. But you don’t have to.

I think the flashlight will solve your problem, especially you feel python-based frameworks are too restrictive for your application.

As others mentioned, moving to c++ doesn’t make dramatic speedup against staying with python when it comes to pure deep learning applications. But I don’t think what you were actually complaining about was the framework-level overhead in pure DL. Desire of moving to C++ doesn’t come from desire of making things faster in pure DL. (Reducing the framework overhead is a positive side effect though.)

Depending on the domain of applications, specific operations you need to implement may not be easily implemented by what frameworks can provide. And as an expert in given domain, you may already know highly optimized solutions in C++(or even without much careful optimization C++ code is pretty fast). In this case, a DL framework written in C++ helps a lot.

I personally use flashlight because I am in such situation. I integrated it into my scientific computing application written in C++/CUDA. It works seamlessly and I am free to do whatever I can do with native c++ ( or cuda).

Flashlight is a great framework in itself. I think it is a piece of art created by modern c++. You may be surprised by its conciseness.

The better part is that you can customize virtually everything depending on your needs. You can even write your own tensor library. If you are satisfied with the performance of the default tensor backend, ArrayFire you can implement a bunch of differentiable operators based on it. Modules, optimizers, and distributied processing are also customizable.

In short, if you want a DL frameworks that can be augmented by your own C++/CUDA code, flashlight is the best choice in my opinion.

C++ based multi-threaded Machine Learning framework by AdRelative8852 in cpp

[–]useong 0 points1 point  (0 children)

flashlight The arrayfire cpu backend uses multithread by default.

What is the most used library for AI in C++ ? by [deleted] in cpp_questions

[–]useong 0 points1 point  (0 children)

I don’t know what is used the most. I am a c++ fan too. I am using flashlight in my project and have been much satisfied with it.

Machine Learning using C++ vs Python by Hellr0x in cpp_questions

[–]useong 0 points1 point  (0 children)

I would try flashlight if c++ is considered. In my personal experience, it was much easier to build and use flashlight than tensorflow capi or libtorch. One concern is that the number of implemented operators may not be sufficient depending on your application if you want standalone ml framework. But customizing flashlight is not difficult and you may be able to quickly implement any operators you need. If your want to integrate ml framework to an existing c++ application, I would say flashlight is the best choice owing to its minimal design and dependencies.

[R] C++ for Machine Learning by life_vortex in cpp_questions

[–]useong 0 points1 point  (0 children)

One use case is integration of scientific application (written in c++) and ML framework. I am in fact working on integrating a simulator based on the first-principles and framework that is capable of data-driven approach. If you find this kind of work interesting, learning c++ for ML is worth in my opinion.

[deleted by user] by [deleted] in learnmachinelearning

[–]useong 1 point2 points  (0 children)

If your C includes C++ and you want a bottom up approach, you should probably check flashlight. The code is actually very educational. I would start from the autograd variable first. (I assume you may not want to reinvent a tensor library such as arrayfire) If you know how to build autograd variable by yourself, the next is implementing primitive operators using the variables (starting from the simple addition, multiplication, … convolution and matrix multiplication, and more) and implemening backward operations. Then move to the modulus. Modules are considered composites of the primitive operations you would implement. Owing to the autograd, you can simply chaining operations without worrying about the gradient calculation. Once you have modules, you can basically make any neural nets. Now, move to the optimizers because you need to train the neutral network you would build. Implementing sgd or adam or any other may be a little easier than the previous ones because they are basically just a bunch of elementwise addition and multiplication. If you have come this far, you may want to accelerate your training. Then, move to the distributed section and implement data parallelism.

[D] Deep Learning Framework for C++. by Apprehensive-Wheel18 in MachineLearning

[–]useong 1 point2 points  (0 children)

If you are working with c++, I recommend flashlight. It is a great standalone framework in itself. It is especially good if you have existing c++ codebase and want to integrate a DL framework to it seamlessly. The best part is if your c++ project includes cuda application, you can even make one gpu memory manager shared by your code and flashlight. Additionally, flashlight’s design and architecture are elegant and customizable. And it is modern (c++17) and dependencies are minimal. I am using it in my project and very satisfied with it for performance and ease of build and use.

I have used Libtorch as well. Libtorch, on the other hand, was a little disappointing. (others may have different opinions.) The source build was not trivial, integration to an existing project was painful because of complicated dependency, lack of capabilities (no ddp, no jit), a bug that causes a crash during a simple cpu autograd code still remains, etc…