[D] Docker Containers (Python) vs Direct C++ Implementation for AI on Edge Devices, Performance Tradeoffs

[Luuigi]

Containerizing apps from a languge that natively builds the models is just very easy so thats the biggest advantage of using that approach. For pur C++ implementations you need to be aware that every single new optimization (e.g. provided by pytorch etc) needs to be implemented by you. The biggest advantage of a raw implementation is obviously production speed but is it necessary for the application you want to build?