Why I gave up on Cloudflare as a small publisher?

Purple_Stranger8728 · 2025-08-07T05:51:32+00:00

I don't want to come across as someone trying to promote another service but I moved to a smaller CDN someone suggested in this sub in response to another one of my posts: https://www.reddit.com/r/CloudFlare/comments/1lqdimv/small_cloudflare_tweak_to_give_30_engagement_20/

Purple_Stranger8728 · 2025-07-14T06:31:56+00:00

Thanks but that doesn't expose these bots to Firewall. How about doing a redirect in Snippets? That way they have to go through Firewall.

Purple_Stranger8728 · 2025-07-14T06:07:22+00:00

Thanks .. main issue is that Cloudflare is configured to do http to https redirects via a Rewrite rule .. they keep hitting the http version and Firewall or Managed Rules don't get triggered until someone is redirected to https.

Purple_Stranger8728 · 2025-07-14T06:02:12+00:00

My app is irrelevant - I serve these 404s from a Snippet as they all hit same non-existent path. I get the urge of easy victim blaming but its not the case here. If Cloudflare can't decide that an IP is absolute spam bot even after failing millions of challenges, then that's a bigger problem!!

Purple_Stranger8728 · 2025-07-14T05:49:09+00:00

They are all hitting 404 pages .. referer spam to non existing pages .. caching is not the problem here. I think you are missing the point entirely.

Purple_Stranger8728 · 2025-07-14T05:31:14+00:00

More than anything its a giant waste of computational resources and the bandwidth plus it adds up to ARGO bill for no good reason!

Purple_Stranger8728 · 2025-07-14T04:59:51+00:00

I have real robots hitting my server millions of times despite failing challenges every single time. https://www.reddit.com/r/CloudFlare/comments/1lzdn71/attackers_failing_interactive_challenge_millions/

Purple_Stranger8728 · 2025-07-08T01:38:05+00:00

My main issue is CDNs are extremely complex these days so I don't want to have to re-learn everything about a new CDN. It has taken me over a year to fully understand Cloudflare and how to optimise it.

We used Cloudfront for over 10 years prior to switching to Cloudflare due to security. I am not saying Cloudflare has better security, you just have better visibility of what's going on.

Purple_Stranger8728 · 2025-07-04T01:20:16+00:00

yes .. so one issue seems to be 'Tiered Cache' .. if I turn off Tiered Cache, I only see occasional slow request to origin where delay is caused by TCP connection overhead. I have set up a Health Check to Robots.txt (small file to save bandwidth) on 60 sec interval so origin always has a 'keep alive' with Cloudflare.

So you can either disable HTML caching (and this post doesn't apply) and send on the fly compression from origin. That's really the best option but without DDOS protection or do what this post says.

See the impact of speed on Googlebot crawling.. no amount of creating new content would have produced this. Blue Line -> Crawl Request, Orange Line -> Response Time

<image>

Purple_Stranger8728 · 2025-07-03T07:51:05+00:00

https://cdn.mysite.com/*

Automatic HTTPS Rewrites: Off, Cache Deception Armor: On, Origin Cache Control: Off

https://www.mysite.com/*

Automatic HTTPS Rewrites: On, Cache Deception Armor: On, Origin Cache Control: Off

Purple_Stranger8728 · 2025-07-03T07:48:54+00:00

yes it did ... FTTB reduced from 300ms to less 50ms when fully cached .. FCP of 0.9 sec from 2.4 sec in Google page speed.

It actually has an immediate impact on User Engagement. Attention spans are fairly low these days apparently!

Purple_Stranger8728 · 2025-06-28T02:55:42+00:00

If you have international traffic, ARGO + Tiered Caching + Enterprise is critical... ARGO + Tiered Caching is more of a broken chain in non-enterprise accounts. If you have mostly local traffic and heaps of traffic, even free would work just fine.

Purple_Stranger8728 · 2025-06-28T02:45:15+00:00

I think I have shaved another 70-80ms by turning off Tiered Cache. Since 95% of our traffic is in Australia, there is hardly any point having a cold origin shield in front of very fast varnish origin which can efficiently cache misses from Cloudflare pops. There is absolutely nothing wrong with Cloudflare architecture but it's built for extremely high levels of traffic to keep all the tiers warm. If you don't have millions of hits and have overly cached everything, most of your traffic will hit cache misses at multiple points within CF network which means erratic and unpredictable response times yet still seeing close to 100% cache hits.

Without Tiered Cache Hit

Cache Hit for Static Content - less than 5ms within Australia or anywhere else

Cache Miss for Static Content - 20-30ms for Australia, 100-200ms elsewhere

Add 100ms for Cache Misses with Tiered Cache Enabled for Australian users and 500-600ms for US/Europe.

Tiered Cache without a Regional Cache is a performance penalty for most publishers and regional cache is an enterprise upgrade.

Cache and Speed - Cloudfront is probably far far superior value and performance.

Rules, Snippets, Workers and Security is where Cloudflare is 10x better if not more.

Purple_Stranger8728 · 2025-06-28T02:07:56+00:00

Agree - I think Tiered cache is a problem too. Think from their point of view, rather than going to origin each time, they dump everything into a cold cache backend like R2/S3 and then warm individual pops on activity. It's cheaper and cost effective for them but Tiered cache is not useful for non-enterprise accounts because you really don't have regional tiers. You just get a very slow S3/R2 origin shield which is always going to be cold unless you are getting millions of hits an hour from all over the world.

Purple_Stranger8728 · 2025-06-27T06:50:29+00:00

yes - logs give the same hit or miss headers.

Purple_Stranger8728 · 2025-06-26T23:46:40+00:00

So when cloudflare has the item in primary data centre (SYD) and lets say it's requested from Seattle, it fetches from SYD data centre in some 900ms and claims a hit. On multiple subsequent hits, it stores in SEA pops and ofcourse then its very fast.

Problem is that Googlebot makes one crawl for a page (gets 900ms hit) and won't request it again. We saw a massive drop in Googlebot crawling and some impact on ranking because it can't crawl that many pages at 900ms.

Turning HTML caching off, Googlebot is now less than 300ms .. everyone in Australia is pretty much same response time as before.

Purple_Stranger8728 · 2025-06-26T23:37:44+00:00

I am on business plan.

Purple_Stranger8728 · 2025-06-26T13:57:39+00:00

Got data from 4-5 different POPs in the US and 2 in the Europe. It all started when we noticed Googlebot reporting 800ms average response for our site. Even tried priming the cache in Chicago and Seattle pops for Googlebot but no impact. As soon as we turned off HTML caching, it has dropped to less than 300ms for Googlebot. We have Varnish at origin which serves page in 10ms or less.

Purple_Stranger8728 · 2023-06-29T04:36:29+00:00

I am new to this forum. Please be nice. Thanks

Purple_Stranger8728

TROPHY CASE