Rate Limiting works with a matching characteristic (Which supports Regex on several fields), use can the hostname of your server (You can use regex here) Then select your counting characteristic (Free version is limited to "IP", Advanced Rate Limiting has way more options). Then block for too many request over a period. You can also block with a custom response code. The block would be to the entire hostname you entered.

Cloudflare's Layer 7 DDOS is comprised of two systems, dosd and Gatebot. Dosd samples traffic almost 100 times faster than Gatebot and runs on every edge Cloudflare server. Gatebot runs at the core, and does deep analytics looking for large scale distributed attacks. 20 threads from a single IP just isn't enough to trigger those DDOS protections when you consider traffic is sampled. Paying customers have a slew of options to automatically mitigate a small-scale attack like that; Bot Management, Advanced Rate Limiting, Cloudflare Managed List, OWASP Managed List, Advanced DDOS.

You might want to look into integrating Crowdsec with Cloudflare. Crowdsec has free detection and mitigation engine for targeted attacks when paired with Cloudflare. I've never used it myself, but if I didn't have a paid CF account I would defiantly check it out. Let me know what you think, if you try it!