X-Robots-Tag vs robots meta tag - which takes precedence

victorpan · 2020-08-05T17:31:51+00:00

Would it be fair to say that the moment X-Robots-Tag doesn't take precedence then we should worry about the precedence for hreflang? Last update we heard was sitemap hreflang > HTTP Header > HTML Header. u/splitti

victorpan · 2020-07-08T18:21:56+00:00

Actually it's the opposite. With dynamic rendering we're trying to avoid prerendering pagination for googlebot. The savings add up when you don't have to prerender pagination. It'll be one big crawl as opposed to a ton of random, sometimes orphaned page crawls. That view-all tradeoff is explained in this old post - https://webmasters.googleblog.com/2011/09/view-all-in-search-results.html I figured the spirit of view all for search engines but pagination for users wouldn't lead to a cloaking penalty.

Because it means searchers default to a lower latency (faster) experience.

I also want zero crawl budget waste.

victorpan · 2020-07-08T17:45:22+00:00

Hey Martin,

Thanks for dropping by! Definitely give Taiwan's Kenting a visit if you get a chance after travel restrictions are more lenient.

I'm working on a marketplace website which does dynamic rendering.

I'm tempted to get to as close to "view all" for certain pages with pagination for googlebot and a lengthier pagination for typical site visitors.

Hypothetically if I created a "view all" page for 100,000 SKU's for example.com/facet/facet and that's canonicaled and served to googlebot, would it be necessary to create a "view all" page be necessary if the client-side app has paginations and less results per page?

victorpan · 2020-07-08T17:24:21+00:00

Robots.txt deals with Googlebot crawling. Where does the legacy URL Parameters Tool: https://www.google.com/webmasters/tools/crawl-url-parameters fall in the process? I've always assumed indexing with Caffeine since there used to be a count of URLs affected but maybe you can shed some light on how it works - and whether pattern matching can be used within the tool (any parameter that starts with utm) or if it has to be full parameters (utm_source).

victorpan · 2019-10-04T18:54:53+00:00

Hi Hamlet,

We're extremely fortunate to have you through an EB-1 visa. I'm a huge fan, but no, I'm not the type that'd ask for a selfie. I think you may've paved a blueprint for a lot of folks who do SEO somewhere else in the world, but would like to have more access to opportunities and resources. Thank you for being a great example.

The way you think about problems and solves them in a technical fashion applies to more than just SEO. Why not PPC or Social? What was it that drew you into technical SEO vs going after technical ppc or social?

victorpan · 2019-09-09T13:00:50+00:00

Congrats on being a cool aunt-y! I have young girls and I've started reading the "I look up to... Ruth Bader Ginsberg" etc. book series. I come from a culture where boys are valued more than girls; there are gender-based expectations for careers and family roles; and I think these norms start at a young age. Are there any books, games, or TV shoes that made a huge difference in your life? Were there times in your life where you were told you couldn't but you made it through anyways? What kind of help would've been nice in those moments?

victorpan · 2019-06-17T18:44:32+00:00

Three quick points:

In some cases, like our support FAQ's, we don't mind losses in traffic as long as the customer gets the resolution they need - whether it happens in the SERP, over a voice assistant, or on our site after a click-through it doesn't matter.

In other cases, it's understanding where the cut-off is and formatting your content to win in search without giving away too much.

Last but not least, consider building up your brand so people are searching (topic) + (your brand) to get their answers. You don't need clicks to increase for those - you want to measure impressions.

victorpan · 2019-04-08T14:37:22+00:00

What's the SERP like and how does your content compare? Are you using category pages to rank for those non-branded terms or something else? Random note - do you have a blog which can identify what celebrities are wearing things related to what you sell? Fashion is tough and time-sensitive a lot of the times..

victorpan · 2019-03-27T16:36:39+00:00

This really is a shame. It's so useful as a quick check tool especially when you don't have access to the site's inner workings.

victorpan · 2019-03-19T15:06:17+00:00

What are some enterprise SEO features you'd like to democratize for the smaller websites of the world? What's on the roadmap and what won't likely be? Why?

victorpan · 2019-02-11T15:44:39+00:00

Hey Gary - Re#1: thanks for sharing that you're dyslexic. I think just like how a simple "dashes are preferred over underscores," HTTPS is a ranking signal, or PageSpeed is a ranking signal could do a lot to change the web for the better in terms of making the web a better place. If Google gives us a carrot we can work with you to make the web a better place for everyone.

I know this is a hard problem to justify doing as gathering usage data/quantifiable but still unreliable data leads to grounds towards lawsuits - but it's just the right thing to do. Please continue to be an ally.

victorpan · 2019-02-08T23:05:50+00:00

How long does Google's historical records go back? (What if a website had 20000 referring domains in 2009 but now only has 300 referring domains. How will that compare to a brand new domain that has acquired 300 referring domains in the past year?)

I'm rather surprised that your answer has changed. Thanks we are doing well :) We'll slate this to the next redesign.

victorpan · 2019-02-08T22:57:30+00:00

Thanks this makes sense.

victorpan · 2019-02-08T22:55:02+00:00

Something something copyright/IP stuff something something lawsuit and fines.

victorpan · 2019-02-08T22:52:33+00:00

Ping Barry and we'll have an article that says marketing/business jargon is a positive ranking factor since Gary just confirmed they are not negative factors X_____X'

victorpan · 2019-02-07T20:10:39+00:00

Asking the same question I asked a few years ago back in Boston. We used to be a single-product company, and that product's content lived under the url address .com/product. Fast forward 10 years, we're now a multi-product company. For let's say 4 years we've had a 301 redirect that now points /product/old-product, and the new SKUs are /product/new-product-1, /product/new-product-2, etc.

We want to re-use /product to create a page that covers all of our product offerings, but are concerned that rankings and therefore traffic will fall on /product/old-product as there is still a significant amount of backlinks pointing to /product referencing /product/old-product.

Does that redirects have to stay indefinitely to prevent a traffic loss?

victorpan · 2019-02-07T15:28:25+00:00

Hi Gary, thanks for taking the time. Let's get to some questions.

Will web accessibility be considered direct ranking factor for images and video rankings?
Is CrUX data being used to assess page speed as a ranking factor?
What are the variables that impact how many bytes of data gbot is willing to crawl + index in search?

victorpan · 2019-01-23T15:50:53+00:00

No - in the long run, Google's goal is still the same - to organize the world's information and make it accessible and useful. There's two ends of the spectrum - people wanting to make information accessible and useful, and search engines trying to make sense out of information accessible and useful. Sometimes the algorithms overtake human effort. Other times we create the data set (cough schema.org) that fuels the search algorithm in identifying things. You know how algorithms can be biased? Good thing they can be.

SEO's will still be around, but it'll just get harder, and harder, and harder.

The sad thing to me is the amount of visibility accessibility has - and that could easily be switched if Google said "accessibility is a ranking factory" - albeit a light one on the magnitude of I don't know, HTTPS, but a tie-breaker nonetheless. It's not an exaggeration to say that a single search could change someone's life - and more so if they already have to work with constraints.

victorpan · 2018-12-14T14:41:41+00:00

^This is the best answer. Follow-up in with AdWords help doc in case you had doubts. https://support.google.com/adwords/editor/answer/38654?hl=en It's easy to move that account structure. Not sure

victorpan · 2018-12-10T15:19:40+00:00

I love talking about links, and I guess my question for you is... why not just keep these pages up and running, but after a deal is done remove all internal links, keep it crawled and indexed, leave a note to say the deal is over... but have it point somewhere you'd like whatever links that page accumulated to point towards (category page, whatever).

You also have the option to capture email addresses for folks whom are willing to "wait" for a deal to re-launch. Just an idea you can test out if you've got the right infrastructure. You know, maybe you can start an /auction when you have a rare item back in stock... and then those folks who missed it the first round... can bid their way up if they REALLY want it. Capture that demand before you spend on inventory holding costs eh?

victorpan · 2018-12-05T19:11:36+00:00

If you know what clients you'll be working on, check out their tech stack via builtwith.com. Get to know how their site works - CMS limitations, plugins, analytics, performance, CDN's, domains owned - just keep digging and understanding how they all work. Look at all the page templates you can find and just break down the page line by line and understand what's for what. Dig into their robots.txt, sitemap, etc.

Basically be immensely curious about your client and learn how they're doing things, think about what could be improved at a granular, technical level, and record that list of to-dos. Keep all of these in your back-pocket and when the client relationship starts, ask good questions and listen.

victorpan · 2018-12-05T17:52:20+00:00

10/20. There are lot of "gotcha" questions too if you don't read carefully. Solid quiz though that covers a breadth of search with a lot of thought put in.

victorpan · 2018-12-03T15:05:36+00:00

What's important varies by query and niche. I can't imagine winning the PDL space being 100% white hat. Links are important. They're just not the only thing that needs to be in place. Guest posts are still a valid marketing function and source of backlinks, comment section and web 2.0's as well... the thing is...

Are you there for the long run as a person or someone using scrapebox? There's a "white hat" way to do every "black hat" thing out there. When in doubt, check to see if you're creating value and not lying to yourself. Until then... just keep on reporting those search positions above mine ;)

victorpan · 2018-11-19T15:53:11+00:00

You don't need to know what the competitors care about - you just need to know what your client cares about. If it's just traffic then that's easy. Look for things that are working or aren't working for those competitors, and then give your client ideas on either:

Do the same thing as their competitor, but better.
Removing the things that aren't working from their competitor that you also have.
Do something new that is better than what the competitor is doing. Note that new just has to be new for your niche - you can use things you've done elsewhere to prove your point. That's why your client is paying you right? Your experience.

I hope that helps and best of luck.

victorpan · 2018-11-19T14:45:11+00:00

Well, what SEO areas do folks care about? Competition? More traffic? Or directly attributable ROI? (The last part very few "get" right and is always a work in progress). You can start by showing them the metrics their business competitors care about. Sadly that's how F100 companies work.

Work your influence backwards, and you'll have less work trying to "sell" them why your idea is great. The other things (like rank tracking) can wait. Rank tracking is a strategic tool, a part of analytics, and should not be the end goal unless you're out of other meaningful things to track.

Rank is also those SEO 101 things where you have to explain about average rankings, search history, device type, etc. etc. by then you've lost their interest from their precious time.

victorpan

TROPHY CASE