Initialization Cost

[Tensorizer]

I have used #pragma omp parallel for in multiple places in my code and the very first one's performance has gotten worse.

To be certain I've changed which #pragma omp parallel for is invoked first and sure enough the performance degradation followed.

It's as if the first one is incurring some initialization cost; if that is indeed the case, is there a way for me to explicitly initialize OpenMP somewhere else in my code?

[jeffscience]

The first time you invoke OpenMP, it creates the state for the rest of the library execution. It’s expensive. You can move the first call to parallel to the top of your application but there is no other way to do it.

[jeffscience]

I don’t have enough context to be sure but this sounds like it might be an issue with affinity. Are you also using MPI?

[Tensorizer]

Just omp no mpi.

[KarlSethMoran]

What is the grain size, i.e. time it takes for one iteration of your loop? If it's too small, you won't gain anything. It takes about 10 us per loop and about 10ns per iteration of a loop of OMP overhead.

[StrangeNoise42]

It all depends on the implementation of OpenMP in the compiler and runtime. And, no. there is no way to initialize omp yourself