On Thursday (November 3rd) of this past week, Linkscape's index updated (in record time - just 3 weeks). New link data's once again available in OpenSiteExplorer, via the SEOmoz API and in the Mozbar. Here are the stats for this latest index update (our 46th index update):

• 43,077,387,028 (43 billion) URLs
• 480,597,551 (480 million) Subdomains
• 105,570,741 (105 million) Root Domains
• 356,255,241,471 (356 billion) Links
• Followed vs. Nofollowed
  • 2.18% of all links found were nofollowed
  • 58.21% of nofollowed links are internal, 41.79% are external
• Rel Canonical - 10.46% of all pages now employ a rel=canonical tag
• The average page has 77.28 links on it (down .19 from last index)
  • 64.86 internal links on average
  • 12.42 external links on average

Since August, we've been struggling with the particularly devious problem of binary files in the index messing up link counts and showing links that Google + Bing probably are not counting. In September's crawl, we put a black list on these files and saw a reduction of ~40% in binary files. This time, we've made even more progress (though it's tough to know exactly how much - we're continuing to evaluate) and you should see a signifcant reduction in these binary files.

In part because of the reduction in these files, processing time for the Linkscape index was reduced, enabling us to produce a much faster index update. However, we're planning in December to produce a much larger index and thus anticipate processing time to rise back up. On the plus side, this will mean a lot more link data. In 2012, we're aiming to reach into the 100billion+ URL index size, closer to what we've heard Bing + Google keep in their main indices (~120-140 billion URLs).

As always, feedback on the new index is greatly appreciated - if you're seeing stuff we've missed, files we shouldn't have crawled or metrics that feel wrong, please let us know. Our engineers would love to hear from you.