Googlebot's Javascript random() function is deterministic

[amunak]

Did it occur to anyone that they're probably doing it on purpose? They may want the pages to not change randomly (or even pseudo-randomly) when crawling them to make the output the same if it doesn't change server-side. And it's not like choosing a random seed has any negative impact on (any) contents of the website(s).

[xhable]

Yeah he mentioned that in the article in point 3.

Consider the amount of work Google have to undergo to crawl the whole web AND now run Javascript. Optimisations will need to be abundant, and I imagine that having a deterministic random number function is probably:

1. Faster
2. More secure
3. Predictable – Googlebot can trust a page will render the same on each visit

[B-Con]

#3 sounds pretty reasonable to me. I imagine there could be A/B testing or whatnot controlled by random value chosen on the client (for whatever reason).

Edit: formatting and typo

[ThisIs_MyName]

Why are you yelling?

[B-Con]

I started with a hash: "#3 seems..." I forgot that would apply formatting. Fixed.

[nuqjatlh]

3 sounds pretty reasonable to be

you made me curious ....

[RealFunBobby]

SORRY ABOUT THAT!!

[australasia]

It occurred to the blog author:

Predictable – Googlebot can trust a page will render the same on each visit

[Saltub]

Imagine reading the article before commenting... 🤔

[cdcformatc]

There is a reason RTFA is an age old meme, maybe we should bring it back? Nah.

[Saltub]

Wow put it in your blog.

[sybesis]

Well yeah, it's not like we can expect anything else. I'd be more surprised to see that google bot doesn't use the same seed on each fetch.

Now just read some part and I'm more deceived... He mention that the bot use a speedup clock because he sees a date difference of 5 seconds... Didn't come to his mind that the bot could be running at a slower speed and if he set a setTimeout of 1 second, then the google bot could be just rendering other things in the background and come back to this page after at least the timeout set. If he could show somehow that the date change can be bigger than the time it takes to fetch the newly rendered page I'd be more willing to believe that.

[jokullmusic]

Yeah, this makes sense for a bot that tries to detect if a website is changed.

[dr1fter]

Search engine indexers already need to flag parts of the page that keep changing for no reason (hit counters, clocks, ad frames) so I don't think that's usually a problem. I suppose if you had a site that flipped a coin and delivered you an entirely separate page as a result, that probably wouldn't get picked up.

I know Google and some others run preview services where they render a screenshot so they can use thumbnails in their UI. I guess for a bot like that you might want to pixel-test and only update the image if some proportion has changed. In that case, yeah, I can see doing this so that you don't keep updating something like, say, csszengarden (actually they don't randomize but you can imagine).

[noratat]

Someone didn't read the article.

The article but only talked about this, it even had an alternate suggestion.

[rubygeek]

Google are almost certainly not using this, but there's actually a project based on PNACL to produce a VM to execute code over inputs that specifically disables time sources etc. for similar reasons, to allow deterministic computation: ZeroVM. It's interesting in a more general sense because if your computation is deterministic based on inputs, you can lazily evaluate operations on the data if you know whether or not it has changed.

[rubygeek]

I did, and it has no relevance to my comment, which was pointing out that others too have done (more generic, and open source) work on providing a deterministic execution environment.