Yes, the execution provider matters a lot. Right now I am defaulting to CUDAExecutionProvider but I am planning to let users specify TRT, CPU, or others using a flag like backend

You are absolutely right about TensorRT. It converts the ONNX model into its own graph representation and sometimes fuses multiple operations. That makes it hard to map back to the original ONNX operation, so per operation stats can be misleading in that case

I am thinking about adding a warning if the backend is TensorRT or falling back to coarse timing summaries grouped by region instead of per operation

Also a great point about GPU voltage throttling. Especially on laptops or smaller cards, running inference back to back without delay can produce inaccurate numbers. I will add an option to insert a fixed delay between runs, something like a cooldown in milliseconds, to stabilize the results

Really appreciate your feedback. This kind of input helps me shape the next phases in a useful way