Today, we are releasing the latest Mozscape index - just three weeks after our last release on 12/21! Mozscape data has been refreshed across all our applications so you can see the latest data in Open Site Explorer, the Mozbar, PRO campaigns, and the Mozscape API.

The top focus of the Big Data team in the next couple months is to get our Mozscape index releases to be more frequent - consistently releasing every three weeks, and then, ultimately, every two weeks. We're utilizing both the high power compute AWS machines as well as our own virtual private cloud setup in our Virginia colocation. At this point, processing in AWS is still slightly faster than processing in our private cloud. However, our top notch Tech Ops team is working closely with us to fine tune our implementation of Open Stack, open source software that allows us to put a virtual layer on top of our fleet of hardware in Virginia. Our own super computers in Virginia should give us even more computing power than we've seen in AWS, meaning faster index processing and more frequent releases for you guys!

Here are the metrics for this latest index:

• 68,291,839,694 (68 billion) URLs
• 512,802,814 (512 million) Subdomains
• 96,918,414 (97 million) Root Domains
• 771,699,931,943 (771 billion) Links
• Followed vs. Nofollowed
  • 2.24% of all links found were nofollowed
  • 56.32% of nofollowed links are internal
  • 43.68% are external
• Rel Canonical - 11.39% of all pages now employ a rel=canonical tag
• The average page has 61 links on it
  •  51.68 internal links on average
  •  8.77 external links on average

And the following correlations with Google's US search results:

• Page Authority - 0.36
• Domain Authority - 0.19
• MozRank - 0.24
• Linking Root Domains - 0.30
• Total Links - 0.25
• External Links - 0.29

This index is a little bit smaller than the previous index, but fairly fresh with the oldest data being crawled late November and the freshest from January 1st. As you can see from the histogram, a pretty big portion was crawled mid- to late-December!

We always love to hear your thoughts! And remember, if you're ever curious about when Mozscape is updating, you can check the calendar here. We also maintain a list of previous index updates with metrics here.