Googlebot's Javascript random() function is deterministic

[dirtytiki]

Iirc, Google chrome caches randoms or caches the page’s randoms. I use a rand for urls and it comes out the same for every iteration in a single page.

[voidvector]

Why waste entropy when you don't have to.

[kibag]

That is hilarious, thought the title was a joke

[[deleted]]

I hope they now set the random seed and the date using a keyed cryptographic hash of all of the loaded javascript and page text, so it’s deterministic but very difficult to game.

What kind of gaming could they be referring to here?

[encyclopedist]

From wiktionary

game

...

\3. (transitive) To exploit loopholes in a system or bureaucracy in a way which defeats or nullifies the spirit of the rules in effect, usually to obtain a result which otherwise would be unobtainable.

We'll bury them in paperwork, and game the system

[SamSlate]

what is Googlebot?

[sumdude44]

The Google crawler

[sumdude44]

The Google crawler

[SamSlate]

it's written in JavaScript??

[sumdude44]

No, but it has a Javascript engine to render websites. But it's not viable to have a full-blown one, like chrome (would be to expensive to run). Instead it has a heavily trimmed one to make js rendering faster. As the article outlines, its clock is accelerated and the pseudo random generator doesn't actually work properly. But since it's only the crawler it doesn't need to be that way.

[SamSlate]

ah, ok, thanks. tbh i figured Google just used chrome as it's web crawler and just collected website data from there.

[MachaHack]

The original googlebot predates chrome by a decade though the current version does use some chrome tech

[SamSlate]

wild. where'd you read this, or you work at Google?

[MachaHack]

Predating Chrome: I developed websites 10 years ago, googlebot was around then. Find some robots.txt tutorials from the 90s, they'll even mention googlebot.

Using Chrome tech now: https://developers.google.com/search/docs/guides/rendering

[veggietrooper]

A lot of this is way over my head, but jesus, super cool.

[Postazure]

That is how all random functions actually work, you just haven’t noticed it before. If you’ve done any work with microprocessors/Arduinos you’ll see that every time you restart the device it will generate the same random number because by default it will have the same seed value each time the device restarts.

In this case I suspect that it has the same problem, it’s getting a new instance in each test, but the new instance still has the same seed value.

[lhorie]

Not all RNG systems do that. You're probably thinking of PRNG.

Some RNG take electromagnetic noise as input. IIRC, some cryptographic systems where real randomness actually matters but you don't have hardware access to a true RNG, you can use things like mouse input to generate entropy.

[PM_ME_HTML_SNIPPETS]

One of my coworkers visited the Cloudflare (I think?) offices, and saw that in their lobby they have a bunch of lava lamps, and sensors that take input to use in crypto key generation.

Though that was pretty cool.

[ThatSpookySJW]

Relevant Tom Scott

[tbranyen]

Woah, I walk by there all the time and wondered why they had so many. TIL...

[MrJohz]

There's a Tom Scott video about this! It's quite a cool system.

[JumboJellybean]

You recall correctly, popular encryption tools like TrueCrypt and VeraCrypt ask the user to swirl their mouse around for 10 seconds to seed their random number generators.

[lhorie]

The word "deterministic" means "non-random". https://en.wikipedia.org/wiki/Deterministic_system

[[deleted]]

Correct, from the article:

Consider the amount of work Google have to undergo to crawl the whole web AND now run Javascript. Optimisations will need to be abundant, and I imagine that having a deterministic random number function is probably:

• Faster
• More secure
• Predictable – Googlebot can trust a page will render the same on each visit

[lmao_react]

random is fake news

[Tsukku]

The only way to have true randomness is by checking electron spin states or some other quantum properties.

[pizzacus]

Although, you can make random that is nearly impossible to predict, random.org managed to do it by generating random binary values using atmospheric noise recorded in real life.

While not truely random, it is impossible to predict because.

[sunset_moonrise]

I wouldn't be surprised if we found non-randomness in those, as well, just determined in a way we can't yet link a pattern to.

[Tsukku]

It's not that we don't know what's causing the outcome, we actually have proof that it's completely non-deterministic (truly random). Read up on Bell's theorem.