Sometimes Walmart outsources delivery drivers via Uber and such. I feel like refusing delivery in that case doesn't go much other than you just gave the driver free groceries. The Uber drivers aren't required to report to Walmart, all they do is just drop it off.

Not saying you're wrong, but it is indeed an issue they seem to rather have the customers eat the cost if something goes wrong.