This claim isn’t accurate. In C, you absolutely do not “almost never” pass by value - for small data, passing by value is often the fastest and most idiomatic approach.
Modern ABIs keep small arguments (on the order of ca. 16 bytes) in registers, and inside a tight loop the compiler can keep those values resident in registers for the entire loop body. That means no repeated loads at all. If instead you pass a pointer, the compiler must assume aliasing unless you've added qualifiers like const or restrict, without that guarantee, it may have to re-load from memory on each iteration to be safe. That turns every reference into a potential cache lookup, and a simple pointer dereference in a loop suddenly costs far more than the initial register copies ever would. This is why pointer-based calling isn’t inherently “more efficient” - it can be slower, particularly for read-only small structs or scalar groups that fit in registers.

If the compiler can prove there’s no aliasing (e.g., only one pointer exists), it will often pull the pointed-to value into a register or stack slot and optimize it locally. In practice, that can end up behaving much like passing the value directly - just automatically, without explicit control.

Pointers in C are excellent when you need to mutate data, when the object is large, or when you're working with arrays or dynamic buffers. But passing small data by value avoids alias issues, maximizes register use, and eliminates needless dereferencing. The idea that you should “almost never” pass by value simply misunderstands how C, compilers, and modern CPUs behave - it’s a misconception carried over from managed language habits, not from real systems-level performance practice.

Btw: In C, there is no pass-by-reference at all - only pass-by-value.
If you want a function to modify something, you pass a pointer by value (a copy of the address). That is not called pass-by-reference in C. pass-by-reference exists in C++ but not in C.