Skip to content

Cyber Sale: Save big on Moz Pro! Sign up by Dec. 6, 2024

Moz news 2 649533a

Use 'What Googlebot Sees' to Find Scrapers

B

This YouMoz entry was submitted by one of our community members. The author’s views are entirely their own (excluding an unlikely case of hypnosis) and may not reflect the views of Moz.

Table of Contents

B

Use 'What Googlebot Sees' to Find Scrapers

This YouMoz entry was submitted by one of our community members. The author’s views are entirely their own (excluding an unlikely case of hypnosis) and may not reflect the views of Moz.

Most of you are aware I recently took a position at Tribune as their in-house SEO manager.

One of the first things I did was get a current state of the union.  After working through the issues of verification for the dozens of sites (another post, perhaps), I realized that 'What Googlebot Sees' for a handful of the domains is 'Privacy Policy'.  Now, this is obviously a red flag for us to add a 'nofollow' tag on the link (that's perhaps another post also), but in addition to that, I found something else.

What's that?  What did I find?

Scrapers . . . lots of them.  You'd think they'd take the time to add a line of code to their bot to remove such a common piece of code as 'Privacy Policy', but . . . nope.

Take a peek at your Webmaster Central console under the 'What Google Sees' section.  Not only will it highlight what you should 'nofollow' but, if you dig a little deeper, you may find some scrapers.  What you do next is your decision.  Options? Take a legal stance, allow them to do it until you gain some more inbound links, reach out to them with an RSS feed (or other linking option), or do something a bit more blackhat, like making your 'Privacy Policy' type links an image with an alt tag for the targeted keyphrase.  ;-)  I'm not going to share what we're doing (though it is not the latter), as that would lead to proprietary information; besides, that type of decision is going to be pretty site-specific, anyway.

P.S. They turned the river green on St. Patrick's Day in Chicago.  Interesting, but not sure if it was worth standing in the cold for 45 minutes with my 4-year old son.  ;-)

Brent D. Payne

SEO Manager

Tribune Interactive

Back to Top

With Moz Pro, you have the tools you need to get SEO right — all in one place.

Read Next

Announcing MozBar V4 - A New Era for Moz’s SEO Browser Extension

Announcing MozBar V4 - A New Era for Moz’s SEO Browser Extension

Oct 23, 2024
Transform Your Marketing Game and Boost Your Income With the New Moz Affiliate Program

Transform Your Marketing Game and Boost Your Income With the New Moz Affiliate Program

Oct 15, 2024
The MozCon 2024 Video Bundle Has Arrived! (Bonus: Our 2023 Videos are FREE!)

The MozCon 2024 Video Bundle Has Arrived! (Bonus: Our 2023 Videos are FREE!)

Jul 24, 2024

Comments

Please keep your comments TAGFEE by following the community etiquette

Comments are closed. Got a burning question? Head to our Q&A section to start a new conversation.