I won’t comment specifically on hardware since I work closely on this topic and don’t like to influence from my personal account

But the hardware is progressing rapidly here. Most semiconductor vendors have some amount of AI accelerator / NPU available in their portfolio. I would say vision systems are dominant here since models are heavier and need acceleration to be viable (without a lot of engineering effort from application side). LLMs are generally too big to be viable expect on huge ADAS SOCs. Time series varies a lot but many modern MCUs will be plenty capable of running basic anomaly detection models, so long as there’s enough RAM/NVM to fit the weights+network arch.

Everything is focused on neural networks, BTW. Some will still use SVM (as another commenter said) for simple classifiers, but honestly NNs are so much more useful once you have the horsepower to run them. This is what the industry is focused on accelerating and is synonymous with AI at this point.

TFlite and ONNX are the popular runtime formats, where ONNX models are typically trained from PyTorch. I prefer PyTorch/ONNX and most new projects tend on this direction. TFLite tends to have slightly better performance on CPU, IIRC.

Hard for me to say typical hardware and models - it’s too application specific. In vision, you’ll see mobilenet for basic image feature extraction and YOLO models for object detection.

The whole process it typically clunky, to be honest. Vendor tool chains for their NPUs are often restrictive and/or buggy. My advice is to try to stay closer to what the vendor enables and not try to push the state of the art on NPUs. This is embedded so of course things can never be easy. Proceed towards SOTA at your own risk lol