
Craigslist Blocks Search Engines
The author's views are entirely their own (excluding the unlikely event of hypnosis) and may not always reflect the views of Moz.
According to a thread at SERoundtable, a few million results in Google (and other search engines) from Craigslist have dropped out due to a change in the robots.txt. The change must be very recent, as I had used Google to search CL just a week before the holidays. It's an interesting strategy from the classifieds giant - If anyone knows the reason, please do tell.
Comments
Please keep your comments TAGFEE by following the community etiquette
Comments are closed. Got a burning question? Head to our Q&A section to start a new conversation.