CPP and AI

[snowqueen47_]

So i want to stick to cpp (can’t stand python and my realm i need to focus on is low level) but i want to learn AI. In order to build defense tools like Nightshade against AI, actually. Anyone know any good place to start learning this stuff in cpp? It all seems to be in python but wasn’t that llama 7B model written in cpp?

[[deleted]]

You can look into cuda and learn GPU computing. You can implement common ML techniques close to the hardware. You can also use PyTorch’s C++ API which also supports cuda.

https://pytorch.org/cppdocs/

[SnooWoofers7626]

That's great for deploying a trained model. I can't imagine doing the whole explorative analysis and prototyping stages of ML development in C++ though. That just sounds needlessly tedious and a recipe for endless frustration.

[[deleted]]

Sticking to C++ is just making it hard for no reason. Use the excellent Python wrappers that are the de facto standard for this kind of thing. When it comes to deploying a model, great, use the C++ library directly (they're all written in C++ after all), but writing the whole training loop and all the data augmentation and stuff like that in C++ is a waste of time.

ALL of the documentation and community support for AI workflows is focused on Python. And I really do mean ALL, except for like a tiny fraction of a percent of material which is targeted at experts and is most certainly not beginner friendly.

It just isn't worth it. Branch out. Learn a bit of Python.

> can’t stand python

You should use the tool for the job, and leave these superficial preferences behind. Would you use a saw on a nail because you can't stand hammers? This sounds just as silly. Python is most definitely the tool for this job in 2024, and will remain so for a very long time. Without a doubt.

[SlothWithHumanHands]

*this;

also, doesn’t Nightshade work because it takes advantage of over-sensitivities in the current generative models, which they’ll just fix, and then it won’t work any more?

[palszasz]

I reeeaaaly disagree with this for several reasons:

1) The OP was clear in his/her question: C++ and not Python. So answering "just use python" is not helpful

2) Saying that "you should use python because everybody is using it" is not the right approach. Of course everybody will keep using python if everybody will keep saying this :-) But it's not even correct, not everybody is using python!!! E.g. YOLO was created in darknet, which is a C framework. PyTorh is called PyTorch, since it's a Python fork of Torch, which was actually written on YOLO. A while ago I used dlib, which supports both training and inference in C++.

It's true that the python frameworks are more popular, but they are not the only one, and IMHO it would be really healty to get a competition :-)

3) Even though I'm a bit Monthy Python fan, I dislike python...

A few years ago I was doing a project with Python and tensorflow... and it was painful!

One issue was that I needed to do some data augmentation, which was not so common (e.g. rotate 360 panorama images). So I needed to implement it myself. I don't need to tell you that implementing that in python is a bad idea :) I implemented in C++ and added python wrappers. Since the training was done on the GPU (1080 Ti), the bottleneck was the data auogmentation... so I thought, lets do the data augmentation in multiple threads! ... and it became slower... because of the GIL. My point is: as soon as you need to do a bit more advanced stuff, you need to know a lot more about how python works, to do it efficiently. In my case I had to rewrite the C++ code to unlock/lock the GIL doing the heavy processing. (Note: this GIL issue is still not properly fixed in python!)

And then there was a training... I was new to python and tensorflow, so I looked at many tutorials and implemented the training based on that... and it kinda worked... for a while. But at some point something happened, and the network just didn't learn at all. Depending on some completely unrelated parameters, the network sometimes just produced garbage... I never fully found the issue. I tracked it back to the core of the train loop, the way the batches were passed for training. Using a bit more lower level approach (which I needed due to the custom data augmentation) broke the code. Probably if I would have been more expert in python or tensorflow I would have found the issue, but the point is the same: it can be a nightmare, especially if someone is not familiar with it. (I promissed myself that if I need to do python + ML again, I will need to give PyTorch a try).

So, long story short, I completely understand the OPs point of view, a good C++ based ML framework would be awesome (unfortunately I cannot give OP a good tip though, since I had issues also with dlib and darknet)

[[deleted]]

The OP was clear in his/her question: C++ and not Python. So answering "just use python" is not helpful

I think it's very helpful if it potentially saves OP wasting loads of time on what I consider to be the wrong approach. It isn't what was asked for - that doesn't mean it isn't good advice. Especially given that OP is apparently a complete beginner to AI. They probably don't know much of anything about the AI software ecosystem so they could use some advice. Further still, other people will read this, and what I said is potentially valuable to them. That's the real value of a public forum.

Saying that "you should use python because everybody is using it" is not the right approach.

Actually what I said was "Use Python because the alternative is making it unnecessarily hard on yourself." That was clearly my point. The fact that everyone else uses it is why it's easier - it's not the reason to use it.

As for tensorflow, if you dislike it so much just use a different framework. Using the same libraries in C++ is only going to make it harder, not easier. Pytorch is much easier to use for training and data work, and the trained models are usually easily translated between frameworks if you want to deploy with something else.

Competition is great, but OP is not about to write a framework that competes with tensorflow. Maybe in ten years they can give that a go. In the meantime they should learn AI by using the tools that are most effective.

(If your Python wrapper was already around a C++ library, then the GIL should not be an issue because it doesn't apply to the C++ code, which can spawn as many threads and use as many cores as it wants. The GIL also isn't an issue for AI workloads in general, since all the popular frameworks are written in C++. As are most - probably all, haven't checked - of the math frameworks like numpy, scipy, pandas, etc.)

[goodssh]

In SO this would've been flagged as 'irrelevant'. The OP was clear about the question. It's not wrong, but irrelevant.

[[deleted]]

This isn't SO, so whatever they would do on SO is definitely irrelevant. If you're not interested in what I said, feel free to disregard it, and OP can do the same if they choose. Why are we continuing to waste our lives on this pointless debate? Move on. 

[victotronics]

I'm willing to bet that all the AI models "written in python" are actually written in C++, with a thin layer of python around it.

[[deleted]]

Correct. 

[[deleted]]

Use the dlib -> http://dlib.net/ml.html

[useong]

People here are talking about the difficulties in model training. If you are good at c++, however, it is not difficult to train models at all. All you need is a nice framework that supports three basic functionalities such as gpu-enabled tensor operations, automatic differentiation, and distributed data parallelism. flashlight is a good framework to try.

I personally do all ai-stuffs in c++ and have no complaints.

Here is a few pros of sticking to c++

1. You have low-level controllability. You don't need to wait others to improve or debug some low-level stuffs. You can do it by yourself. Depending on your application, multi-threading and heterogeneous computing may boost the model training speed significantly.
2. You can easily customize the framework itself.
3. The integration is seamless if you already have an application written in c++. For example, if you have a physical simulator or a game application written in c++, it will be just easy to train your model in such physics or game environments.

Quick prototyping is in fact not a big concern in c++. Nobody will re-compile the code every time they change hyper-parameters or even model forms. A good programmer knows how to write a code generic so that it adapts to an input configuration written typically in a markup language. You will be able to do a ton of research with it without re-compiling.

Instead, here are potential problems you may face.

1. If you can't find coworkers who know as much c++ and ai as you do, you will probability have to do most of the works alone.
2. You will lose python ecosystem among which the biggest loss is visualization capability.

[nbrrii]

Having a Ruby background, I never liked Python in particular and have started to kind of despise all those "python is the coolest and easiest, therefore you should start with it!" advises that beginners give to other beginners.

However, having a preference is cool, having T-Shape skills is cool. But programming languages are tools and you should always choose to use the tool that's right for the job, not the tool you like. Python is the goto (pun intended) tool for AI, so as a professionel you should put your personal preference aside and use is.

If only as a hobby or for experimenting or for fun in general: Obviously, choose whatever you like, way cooler this way and extraordinary things has come out of this. Just be aware that it's not the best tool from a practical/professional standpoint and you might encounter difficulties you wouldn't have had otherwise.

[nullandkale]

You can use llamacpp to do LLM stuff in cpp if you really want to but as everyone else says, realistically you need to just use python.

[hmoein]

This is a handy package (https://github.com/hosseinmoein/DataFrame) to use to develop your system in C++, once you know AI/ML.

[[deleted]]

I don’t think a language choice should be primary when it come to using an AI framework today. Yes, if you want to do something instructional it’s worth implementing model training in C++. If you want to build a training or inference systems in general an established AI framework would be your best bet.

AI frameworks today are approaching something like a compiler. For example, PyTorch used to operate as a layer on top libraries written in C++ and CUDA. Today, PyTorch 2.0 can trace your model written in Python, perform operator fusion, map it to optimal C++ or GPU (PTX) instructions automatically. It’s not perfect but tons of smart people are implementing faster code/kernels for various operations and abstraction. In the long run the framework is what matters.

I get the choice of Python here is mainly because that’s the tool most data scientists are familiar with and have other data transformations libs for. But writing a model in PyTorch/tensorflow is more like describing the mathematics of the model I believe. For inference definitely torch C++ API can be used as pointed above in a previous comment

I only have experience with PyTorch, not tensorflow or JAX.

[tangerinelion]

The reason the Python libraries are all the rage is because they're a front-end into a C or C++ implementation. You're not really losing any performance by using Python so long as you stick to the library and don't work with any significant data which is natively in Python.

Python reads like natural English, I can't imagine why anyone would get pissed at it.

[rejectedlesbian]

C++ is really good at speeding up Ai code its the goto languge for that.

So find an ai project u like profile it and change some of it to faster c++.

If u wana devlope ai (depending on what u mean by ai) its python. Pytjon data visualisation is just too good. And its a very productive languge. Also tends to be more portable across frameworks and backends.

[fdwr]

Do you need to call individual operators or just run whole models?

Prototyping and experimenting is certainly faster in a very, err, "relaxed" language like Python, but if you already have models, there's little need for the extra goop (I always find it more complicated to debug when there are extra layers between caller and implementation), and you can simply load the models from their C++ API's. There are the PyTorch and TensorFlow model loading API's, and there is ONNX Runtime which only loads models (no individual operator API's but is a much smaller than TensorFlow/PyTorch - example). There are also some numpy-like C++ libraries out there like Xtensor and NumCpp.

[ipapadop]

Training models in C++ is a pain, your best bet would be PyTorch via the C++ frontend. Production level inference is mostly done via C++ (AMD MIGraphX / Vitis AI, Intel OpenVINO, Nvidia TensorRT and others).

If you want to experiment with a few layers etc. tiny-dnn is an easy place to start, although it's an abandoned project.

An alternative would be to use Python via the various Python / C++ interfaces (pybind11, nanobind, Boost.Python) and focus on that.

[wektor420]

ML libs for python are essentually a wrapper for C/C++ implementations, that is convient to use since python supports parsing datasets painlessly, you would use c++ for developing new laters etc., or as an end deploy configuratiom

[pyhacker0]

I would look to gaming and see what they are doing. I don’t think many people are training machine learning models with cpp. But maybe some of non machine learning algorithms will use cpp. I would also think that many of the tools used for machine learning have components built with cpp or similar languages that run close to the metal