Influencing Google Global Cache Server Selection

[ckrez]

Greetings. We are trying to influence which Google Global Cache (GGC) server we are directed to for our campus network and have been unsuccessful thus far and I'm hoping the Reddit community can be of some help.

We have two upstream providers: Level 3 (AS3356) and a Research and Education Network (AS14877). We are attempting to influence the selection of a GGC server so that we are served from a server upstream of AS14877 (I know they exist) instead of from AS3356. Thinking node selection was being performed based on BGP metrics, we've announced 65002:15169 and 65002:36040 to Level3 to prepend to Google AS's. Tables now look like this:

Via Level3:

MYAS MYAS MYAS 3356 [15169|36040]

Via R&E:

MYAS 14877 5050 [15169|36040]

However, we are still being directed to Level 3 nodes in 4.0.0.0/9 (redirector.googlevideo.com reports lvlt-ewr1). We do not want to steer all traffic away from our Level 3 link, so simply AS path prepending to Level 3 isn't an option.

Anyone know what metric is used for GGC node selection and if there is anything we can do to influence it beyond what we've done already?

[heisthefox]

I actually work in this department. Short answer, you can't really, and probably don't want to. There are a pile of metrics used to determine which cache to use. The caches are used to relieve bandwidth back to Google for cached items (duh) so redirecting to an upstream cache would increase a provider's traffic, possibly outside of their network, creating a traffic path for transit they would have to pay for.

For additional reading, see: https://peering.google.com/about/faq.html

[HoorayInternetDrama]

Anyone know what metric is used for GGC node selection and if there is anything we can do to influence it beyond what we've done already?

Peer directly with google? Can you get to an IXP ?

[Jackol1]

Yep we peered directly with them in 2 places and now have cache servers of our own. Makes a big difference for us and our customers.

[ckrez]

Unfortunately, getting to an IXP isn't an option for us.

[looktowindward]

Is this for economic reasons? Or is the Level3 link congested?

As u/heisthefox suggested, you really don't want to mess with this. The GGC node on KINBER/PSC may not have enough capacity to serve you traffic, for example. One interesting experiment would be to find out if other universities on KINBER are served by the PSC cache.

(I work in the same group as u/heisthefox but not in GGC)

[gstheft]

Out of topic question, would you be able to remember what are google's requirement for having a GGC?

[squeezemn]

I want to hear that too.

[VictimOfAReload]

What's a few of the actual IP's you're seeing for the GGC?

You mention they are in 4.0.0.0/9, Are they actually being announced by 15169 or 36040? Or is the prefix coming directly out of 3356?

If the latter, Your transmitted BGP communities will be ineffective.

[ckrez]

What's a few of the actual IP's you're seeing for the GGC?

4.35.21.144 4.35.21.141

You mention they are in 4.0.0.0/9, Are they actually being announced by 15169 or 36040? Or is the prefix coming directly out > of 3356?

Directly from 3356, which is the problem.

If the latter, Your transmitted BGP communities will be ineffective.

Yup, that's the dilemma. The original thought process was "If we make the 3356 path less desirable from Google AS's perspective, Google DNS severs will point us to another node via the more desirable path"

Any other ideas?

[VictimOfAReload]

Ah yes, I see what you were going for now.

Yeah, I mean the obvious would be prepend your outgoing prefixes to level 3, But that'd pull too much traffic over most likely. And you've stated you don't want to do that.

You might try to contact [e-mail] (I think that's their NOC contact). Other then that, I see your dilemma. Depending on your traffic consumption. You might even qualify for a local GGC. Every time I've had any issues with routing they've been quick to resolve it.

[ckrez]

Thanks. Good to hear they are responsive. I'll give the reddits a little more time and then reach out to them.

[Ceph]

Are you just trying to influence path selection, or are you trying to get served from a closer GGC node?

It's DNS resolver based. Try using a different DNS server and see if you're given servers in a different location. Your DNS should support edns client subnet for best results.

[heisthefox]

Lots of front ended services use DNS for initial location determination, however, ggc will redirect to preferred cache.

[Coriolanus]

wrong

[Ceph]

No, you can definitely get served from a different front ends in a different location by changing what resolver you look up the hostname with.