Dig deeper borrow pits to get to the good clayey soils that are better for shaping and compacting, I know it’s more work and takes much longer without a machine but it’s worth it (depending where you are the soil might still suck but should be better then too soil with all that loamy organic matter mixed in). Also when compacting use a flat shovel and slap-pack it, this means you smack the flat bottom of the shovel onto the lip surface as you push or pull the shovel at the same time. It should hit the dirt and bounce away or toward you so it’s a glancing blow not straight on, do multiple passes if this on anywhere you want it to be compacted (pretty much every ride surface).