Use 'What Googlebot Sees' to Find Scrapers
This YouMoz entry was submitted by one of our community members. The author’s views are entirely their own (excluding an unlikely case of hypnosis) and may not reflect the views of Moz.
Most of you are aware I recently took a position at Tribune as their in-house SEO manager.
One of the first things I did was get a current state of the union. After working through the issues of verification for the dozens of sites (another post, perhaps), I realized that 'What Googlebot Sees' for a handful of the domains is 'Privacy Policy'. Now, this is obviously a red flag for us to add a 'nofollow' tag on the link (that's perhaps another post also), but in addition to that, I found something else.
What's that? What did I find?
Scrapers . . . lots of them. You'd think they'd take the time to add a line of code to their bot to remove such a common piece of code as 'Privacy Policy', but . . . nope.
Take a peek at your Webmaster Central console under the 'What Google Sees' section. Not only will it highlight what you should 'nofollow' but, if you dig a little deeper, you may find some scrapers. What you do next is your decision. Options? Take a legal stance, allow them to do it until you gain some more inbound links, reach out to them with an RSS feed (or other linking option), or do something a bit more blackhat, like making your 'Privacy Policy' type links an image with an alt tag for the targeted keyphrase. ;-) I'm not going to share what we're doing (though it is not the latter), as that would lead to proprietary information; besides, that type of decision is going to be pretty site-specific, anyway.
P.S. They turned the river green on St. Patrick's Day in Chicago. Interesting, but not sure if it was worth standing in the cold for 45 minutes with my 4-year old son. ;-)
Brent D. Payne
SEO Manager
Tribune Interactive
Comments
Please keep your comments TAGFEE by following the community etiquette
Comments are closed. Got a burning question? Head to our Q&A section to start a new conversation.