The latest Mozscape index is now live - just two and a half weeks after our last January index release! The team continues to improve the frequency of our release dates as we move closer toward consistent 2 week release cycles. All Mozscape data has been refreshed across all our applications so you can see the latest data in Open Site Explorer, the Mozbar, PRO campaigns, and the Mozscape API.

Over the past few months, the Mozscape processing team has put a lot of work into removing the unpredictable variables that led to several missed release dates in 2012. Switching to the large super computing reserved instances on AWS has led to much smoother processing cycles and less machine failures to recover from. The team has also been able to isolate several steps in our processing software that could benefit from some optimization. These updates have led to a significant amount of time saved during processing due to fewer disruptions and more efficiently running software.

As a result of all these changes the team has implemented over the past few months, four of the last five indexes have been released within 3 weeks or less of each other - which means fresher and more frequent data for you!

You can see from the metrics below, there is still a significant increase in the number of subdomains crawled in this index. The increase of subdomains is due to our crawlers discovering a small number of root domains that have a substantial number of subdomains associated with them. These subdomains have very low authority, so they won't affect the metrics in the index, however, the numbers of subdomains has increased again. 

Here are the metrics for this latest index:

• 77,093,101,764 (77 billion) URLs
• 4,263,496,373 (4.2 billion) Subdomains
• 160,052,583 (160 million) Root Domains
• 840,437,839,728 (840 billion) Links
• Followed vs. Nofollowed
  • 2.26% of all links found were nofollowed
  • 56.49% of nofollowed links are internal
  • 43.51% are external
• Rel Canonical - 15.05% of all pages now employ a rel=canonical tag
• The average page has 73 links on it
  • 62.66 internal links on average
  • 10.27 external links on average

And the following correlations with Google's US search results:

• Page Authority - 0.36
• Domain Authority - 0.19
• MozRank - 0.24
• Linking Root Domains - 0.30
• Total Links - 0.25
• External Links - 0.29

This index took a total of 11 days to complete processing so a fairly large portion of the data was crawled in the second half of January. The oldest crawl data will be from late December, but the bulk of this index was crawled after January 15th.  

We always love to hear your thoughts! And remember, if you're ever curious about when Mozscape is updating, you can check the calendar here. We also maintain a list of previous index updates with metrics here.