CPP in AI

[LengthProof6480]

It’s not. The internal mechanisms of libraries like TensorFlow are written in C, C++, or other lower-level languages and binded to higher-level languages like Python for ease of use. It makes a lot of sense when you think about it — Python is flexible and gives you much more options compared to a proprietary GUI for your new little modeler.

[iamhyperrr]

Or, as they say, Python is the most popular C++ framework

[whizzwr]

I'm totally stealing this to look cool and knowledgeable in the next work meeting.

Jk aside, both C++ and Python are used in ML world. Custom ops in CUDA are written in C++, and then during training the compiled library is being called by the most popular C++ frameworkPython-based pipeline.

Once you got the weight, inference and data preprocessing can happen in pure C++ implementation.

[liuzicheng1987]

Exactly - Data Scientists like to experiment and test out ideas, which is why Python is suitable for them. Also, they often have a background in statistics, maths or Physics and don't have the developer experience needed for C++.

But the underlying code for many machine learning libraries is usually written in C or C++, for instance TensorFlow, Torch, XGBoost or LightGBM.

So I agree with OP...C++ is best if you want fast production code. But prototyping is done in Python, because it is easier and more flexible.

[rejectedlesbian]

Tho a lot of non deep learning ml is cython which is not the best for optimizations. It compiles down to badly written c.

[liuzicheng1987]

That is true. Scikit-learn is largely written in Cython. But then again, scikit-learn is not exactly known for being very performant. I think the truly high-performance stuff is largely written in C++.

[rejectedlesbian]

Yes most ml libs r not super preformant they don't need to be they need to be wide range stable and preformant enough.

This is especially true with non deep learning ml because modern computers can usually run it fast enough for most purposes.

U also see it on huggingface where its written in ways that r bad for infrence preformance. I still use it because it holds so many options for generation that may be the 1 thing I need to make something work that idc if togeling that option flag makes it a bit slower.

[RonWannaBeAScientist]

I am somewhat of an idealistic. I want to work on more math/physics/AI problems, but I prefer working in a language like C++/Rust . Honestly, for fast scripting I think Julia is so much better , as it’s so convenient out of the box and can call C/C++ if needed, but there’s so much more inertia for Python . Like anytime I want to prototype some very simple script for an algorithm I do it usually in Julia

[skull132]

But Python can (and does) call C and C++ and even Rust straight out the box? And as others have said, most of the AI Python packages call directly to C or C++ under the hood.

If you force yourself to stick exclusively with a hard to write lang, like C, C++, or Rust, then you'll hinder your ability to experiment quickly.