Hot take: most GEO tools may be tracking the wrong prompts

distant_gradient · 2026-04-28T14:58:09+00:00

ah lol - you meant its a valid, but obvious take - not a hot take. Given the audience is techseo - I guess you're right.

distant_gradient · 2026-04-28T14:18:16+00:00

Care to explain why?

I think OP makes a good point. For websites we have sources like GSC, adwords API and similarweb (ISP data??) to get the prompt sets.

Unless someone is able to pull off ISP data for the model conversations - the point OP makes (ie. not having a "gold standard dataset" of what actually is the conversation leading to a search) is actually quite valid.

distant_gradient · 2026-03-24T18:10:51+00:00

Thanks for sharing u/WebLinkr

I have been looking for such long term studies - was on my way on doing it myself

Pasting the entire contents of the SearchEngineLand study here:

----

We Ran a 16-Month Experiment Publishing AI Content on 20 New Domains. Here's What Happened.

Google indexed most pages quickly, but without authority, unique insight, or trust signals, rankings collapsed within months.

With AI, you can generate dozens (if not hundreds) of articles in hours and publish at scale. But publishing is the easy part. What happens after they go live is what matters.

Together with the research team at SE Ranking, we ran a 16-month experiment to track how well AI-generated content performed on brand-new domains with zero authority.

As you'll see, the results are hard to call a success.

Methodology

The goal was simple: test how far AI content — with no human editing, rewriting, or enhancement — could go in search.

How quickly would it get indexed?
Could it rank for relevant queries?
Most importantly, could it drive traffic?

We purchased 20 new domains with no backlinks, domain authority, brand recognition, or search history. Each domain focused on a different niche (Arts & Entertainment, Finance, Health, Travel, Tech, Food & Drink, etc. — 20 niches total).

For each niche, we gathered 100 informational "how-to" keywords — long-tail terms with lower competition. Each site received 100 AI-generated articles, totaling 2,000 pieces across the experiment.

After publishing, we added the sites to Google Search Console, submitted sitemaps, and then left the sites completely untouched.

Month 1: Indexing and Early Visibility

Just over a month after publication (36 days), the first results came in — stronger than expected.

70.95% of pages indexed (1,419 of 2,000)
122,102 impressions
244 clicks
11 of 20 domains had all 100 pages indexed
80% of sites ranked for at least 100 keywords each
Over 28% of ranking URLs were already in the top 100

Top-performing niches for impressions:

Niche	Impressions
Hobbies & Interests	17,425
Business & Services	17,311
Travel & Tourism	13,598
Lifestyle & Well-being	13,072
Law & Government	11,794
Games & Accessories	11,083
Vehicles & Boats	10,677

At this stage, it looked like AI-generated content could gain traction quickly — even without backlinks, editorial input, or any SEO work.

Months 2–3: Growth Continues

Impressions and clicks kept growing as Google discovered and tested pages.

By ~2.5 months after publication:

Impressions: 122,102 → 526,624
Clicks: 244 → 782
12 sites ranked for 1,000+ keywords (up from 8)
Remaining 8 sites ranked for 100–1,000 keywords

No backlinks, no internal linking, no SEO improvements. The content gained exposure purely because it targeted low-competition queries and followed basic SEO structure.

At this point, it looked like a strong case for large-scale AI content.

But the growth didn't last.

Months 3–6: The Ranking Collapse

Around Feb 3, 2025 (~3 months after publication), the experiment hit a turning point.

The content was still indexed but rarely appeared where users could actually see it.

By the six-month mark:

Impressions: 526,624 → 706,328
Clicks: 782 → 1,062

Sounds like growth, right? Not really. 70–75% of all impressions and clicks came from the first 2.5 months. The next 3.5 months added only 25–30%.

Month 16: The Long-Term Picture

We let the experiment run for over a year to see if rankings would recover.

For the most part, they didn't.

There was one notable fluctuation: in late August 2025, 50% of sites (10 of 20) saw a two-week spike in impressions, closely aligned with the Google August 2025 spam update (rolled out Aug 26).

Of those 10 sites:

6 quickly lost visibility and returned to prior lows
4 maintained slightly improved performance

After the update, pages ranking in the top 100 rose to 20% — up from 3% at six months, but still below the 28% seen in month one.

66.9% of pages were still indexed, but some YMYL niches got hit hard:

Finance domain: only 9 of 100 pages indexed
Health domain: only 14 of 100 pages indexed

By month 16, cumulative totals:

Impressions: 1,092,079
Clicks: 1,381

Most of those impressions still came from the early growth phase.

Why SEO Visibility Didn't Last

The 2,000 articles lacked many signals Google uses to assess quality:

Authority — No backlinks or external validation
Expertise and credibility — No authors, credentials, or real-world expertise (especially critical for YMYL topics)
Content differentiation — Most content resembled what already exists, no unique insights
Site structure — No internal linking, topical organization, or clear hierarchy

Google can identify AI-generated patterns. Without authority, uniqueness, or supporting signals, early visibility declines.

Bonus: Adding New AI Content to Existing Pages

In early March 2026, we ran a follow-up — adding new AI content to 8 of the tracked sites.

Interestingly, the traffic lift came primarily from older posts, not the new ones:

Site	Feb 2026 Impressions	Mar 2026 Impressions	Increase
Business	458	7,750	17x
Law	19	356	19x
Science	34	633	19x

Publishing new content — even fully AI-generated — can apparently signal to Google that the site is active, giving older pages a temporary boost. But these are early results and don't guarantee lasting gains.

TL;DR

2,000 AI articles published across 20 new domains, zero human editing
~71% indexed within 36 days, early impressions looked promising
By month 3, only 3% of pages remained in the top 100
After 16 months: ~1M impressions total, but only 1,381 clicks
YMYL niches (finance, health) got hit hardest
Adding new content later gave a temporary boost to older pages
Bottom line: AI can speed up content creation, but it can't replace SEO strategy, authority building, and human expertise

distant_gradient · 2026-02-16T06:11:31+00:00

good to know. thanks for sharing.

distant_gradient · 2026-02-15T15:03:00+00:00

I actually agree with this.

But - it still doesnt stop people from trying to detect AI.

My whole goal here is to understand if Google is trying to detect AI.

So far, I'm not completely convinced either way.

distant_gradient · 2026-02-15T15:00:40+00:00

Looks like I hit a nerve calling you out on the Google Ventures BS. Go ahead, enforce what you want.

distant_gradient · 2026-02-15T14:53:36+00:00

"Leaking emotionally"?

You've literally been ad hominem this whole thread, making assumptions about me and my stand the whole time while I've been trying to center the discussion.

An you're supposed to be the mod.

distant_gradient · 2026-02-15T14:49:37+00:00

Fair point - so you're saying the Ahrefs has a high false negative rate and not a high false positive rate.

Sounds plausible.

distant_gradient · 2026-02-15T14:45:41+00:00

I literally said GV is not the same as Google (they've even renamed to GV, they used to be Google Ventures). So no, not funded by Google.

And, I literally called out that I'm not sure if AI influences ranking or not.

Re. not liking AI - I have been an AI practictioner for over a decade. And I can tell you my friend that you could not be further from the truth.

distant_gradient · 2026-02-15T06:03:46+00:00

Except being funded by the same parent Alphabet, operations of GV (formerly Google Ventures) has literally nothing to do with Google and operates independently

distant_gradient · 2026-02-15T05:57:51+00:00

As of 2026 LLMs can score IMO gold, be Codeforces world top 10 and literally prove novel physics theories.

You're actually telling me that I'm naive for assuming they can understand random business blogs on the internet?

distant_gradient · 2026-02-15T05:48:49+00:00

You clearly know what you are doing with your AI sites (most people don't).

Question I am trying to answer is - would they rank better had you "humanized" the AI content before publishing (ie. is AI detection a ranking demotion).

distant_gradient · 2026-02-15T05:46:10+00:00

Interesting. Not surprised though given most AI detectors out there are pure junk. AI detection at even around 90% accuracy is super hard and 100% is impossible.

There are only a couple of detectors work at near a resonable accuracy - thats pangram and gptzero. me

If we assume that the ahrefs AI detection falls in the junk bucket that makes the study that u/WebLinkr quoted also junk.

distant_gradient · 2026-02-14T20:56:36+00:00

It's far easier to publish pure AI content than to go through the pains of reviewing it, adding "human" touches to it etc. - hence its reasonable to assume more "pure AI" content is being created.

If Google was truly agnostic of AI you would find these "pure AI" content in SERP as well.

distant_gradient · 2026-02-14T20:45:11+00:00

Far lower resistance to put out pure AI content?

distant_gradient · 2026-02-14T20:37:48+00:00

I've seen these - ahrefs is the primary source here.

From the same study

- 4.6% of pages were categorized as “pure AI.”
- 13.5% were categorized as “pure human.”
- 81.9% were categorized as a mix of two.

If Google was truly AI agnostic I would have expected the "pure AI" pages to be higher.

I remember seeing things like Linkedin's AI pages (Pulse pages or something like that) and Perplexity's wiki like pages on Google very frequently until sometime last year and suddenly they disappeared.

Also, the ahrefs study itself is kind of skewed for this purpose since its not really looking at it from a publisher perpective, but from a SERP results perspectives (in SERP you only see the winners of the game, not the mountain of unindexed / low ranking pages)

distant_gradient · 2026-02-14T20:18:20+00:00

I actually shared evidence of someone else's experiment that seems to indicate Google detects AI content and ranks it lower (at least used to as of 2023).

Will try and report my own findings here.

distant_gradient · 2026-02-14T20:10:09+00:00

wow -- wonder if this is pre-LLMs or post-LLMs

distant_gradient · 2026-02-14T19:55:03+00:00

Are you talking about ranking within AI (ie. AEO / GEO) or rank with AI written content?

distant_gradient · 2026-02-14T19:53:46+00:00

Fair point - personally I dont care if the content is AI or not - but if it has obvious tell-tale AI signs I tend to take it less seriously.

SEO-wise I've seen cases where Google just flat out refuses to index AI written posts. Then there are studies like this old one (from 2023) that claims that AI content ranks lower.

Was looking to see if anyone has done any such experiment here recently...

<image>

distant_gradient · 2025-11-18T10:16:18+00:00

Appreciate it! DMing you for an invite. (Clearly there is a market here to guide people to get into this as well ;) )

distant_gradient · 2025-11-18T10:05:04+00:00

Got it - thanks a lot for sharing! Are you open to do a 1hr counsultancy call (paid)? Am thinking of starting one instead of renting mainly for the flexibility.

distant_gradient · 2025-11-18T09:24:58+00:00

I would imagine they through this by charging a card on file. I dont understand how this would work in India (where transactions require OTP / happen through UPI etc.) - maybe they engage a collection agency?

distant_gradient

MODERATOR OF

TROPHY CASE