This week we are thrilled to have a special guest joining us for Whiteboard Friday: Duane Forrester, a top-of-the-line SEO who went over to the other side of the fence and now works at Bing's Webmaster Project as their Senior Project Manager. Duane's the one you'll see throughout their blog, and if you have a feature request or any questions about Bing's Webmaster tools, he's your man. Duane joins Rand to discuss a multitude of search topics, including Bing's Webmaster Tools suite, the metrics used and displayed in their tools, and some exciting and extremely important news about Bing's use of quality thresholds for sitemaps. Check it out, and let us know what you think in the comments below!

Video Transcription

Rand: Howdy, SEOmoz fans. Welcome to another edition of Whiteboard Friday. This week I have another very special guest for you. Duane Forrester used to be one of the chief SEOs for Microsoft, architecting all the things that Microsoft had to do about SEO. Duane, you felt the webmasters' pain, you felt the SEOs' pain, and now they handed you the keys to the Ferrari and you are running Outreach for Webmasters for Bing, the search side of things. You switched sides. It's amazing.

Duane: Yeah, totally.

Rand: First off, thanks for being here. I really appreciate having you.

Duane: Thanks for having me over.

Rand: Second, tell me what this change has been like for you, to go from SEO guy to search engine guy.

Duane: Yeah, it's been kind of intense. I spent over a decade of my life as an in-house search SEO, and this is a dramatic change. Seeing things from the other side gives me an entirely new perspective. Some things are very validating. Other things, I'm slightly embarrassed by.

Rand: That's a good combo.

Duane: Yeah, and it's humbling is what it is. So, part of what I'm keeping in my mind as I go into work every day and I work on Webmaster tools and I work on bringing new ideas and content forward is this idea that I was there, and the people that are there now are fulfilling a very important role for their business. They need as much help as they can get. So I truly am the eyes and ears of it. It's amazingly exciting.

Rand: It's great for you to have that background rather than . . .

Duane: Completely.

Rand: . . . an attitude of antagonism potentially towards people doing SEO. You've got one of empathy. I think that's wonderful.

Duane: No, my antagonism is directed internally toward everybody I work with because I want all this stuff and they keep bringing me in.

Rand: So on that front, let's talk about some of the stuff that you've already gotten into the product. The last six months have been phenomenal. I just logged in for the first time in about 30 days, maybe 20, 30 days, and there's tons of sweet new stuff. Talk to me first about this Index Explorer thing. This is kind of spiffy.

Duane: So, Index Explorer for you Mozzers who have been watching, you'll know this. Index explorer is an area, if you come into Webmaster tools and you actually look at the tab called Index across the top, on the left-hand side at the top of the navigator, you'll see Index Explorer there. I've known about Index Explorer for a few years now because it's something that I can use as an internal employee to kind of peer into Bing's index and see what's there.

It's a really handy thing for us, because when I was SEO for MSN, it was at vast scale. So, if I had an indexation problem, it was a magnitude of orders. Now, actually everybody who has access to Bing Webmaster tools sees a version of that, applicable to their own domains.

Rand: So this is like the internal tool pointed externally and you can see the stuff that applies to you.

Duane: Right.

Rand: So, I can go in, I mean we just did this with SEOmoz.

Duane: Yeah, exactly.

Rand: We looked at, here's SEOmoz.org and you click this little drop down and it's got all the folders in there. So now I can see slash blog. Then I can see any individual page, for example, post.html. That's not actually our structure. Then if I click on this, it pops up a little box and I've got links and I have anchor text.

Duane: Exactly, yep.

Rand: I can export that.

Duane: There are actually a couple of really neat features in here. So, what we do is in this area we give you the ability to come in and take a look at something. Maybe you're looking at and you're saying, "Jeez, you know, I had a duplicate content issue there, and I've since installed a rel=canonical to solve my issue. But that's still in the index, so I need to remove it." You can actually click a button right here and block that. That tells us, "Hey, you know what? You shouldn't bother with this anymore." Thank you very much. We move on from there.

Rand: How does that act? Does that act like a robots.txt blocking?

Duane: Essentially, it basically tells us that, okay, if you, the site owner, are saying don't go there, then we're going to honor that, because, quite frankly, if everyone took that kind of care and attitude toward maintaining the website, it lessens the resource load on us.

Rand: Cool.

Duane: By allowing this little button, this little button . . .

Rand: That little button right there.

Duane: Now orange. It makes a big difference for us. And we give you the option, you can block an individual URL, or you can block a folder if you want. You may look at that and say, "Oh, you know what? I don't need anything in this print folder. Block that."

Rand: And now I don't have to write to robots.txt if I don't have access?

Duane: You still should be writing to robots.txt. However, there are cases where people won't have access to it, and this is an opportunity for them to still communicate the need.

Rand: Sure, absolutely.

Duane: Another area that's in here that's really cool is we're going to show you how many links there are to this particular URL. You'll see a little number here. The one we looked at was 198. And then if you actually click on where it says "pages linking to this page," click on that, it opens up this really nice, beautiful pop-up, and that actually contains all of the URLs, the domains that are linking to you, the URL that it's on, and the anchor text that's being used to provide all those 198 links to this individual URL on your website.

Rand: And you're showing these, if I recall here, you're showing them in sort of a date order of crawling . . .

Duane: Right.

Rand: . . . and you're showing up to 20,000 per URL.

Duane: Exactly, exactly.

Rand: That's a lot of link data.

Duane: That is a lot. For most small, medium, and even a lot of large websites, that is going to be far in excess of what they're going to need. That is on a per URL basis, so it's not 20,000 overall. It is 20,000 applied to this individual URL. The next URL gets another 20,000, and another 20,000 and so on.

Rand: This is potentially a lot more data than I can see from any existing source at least about my own site.

Duane: Exactly.

Rand: Because Webmaster tools in Google has a sort of a limit on, a smaller limit and it's across the whole site.

Duane: Right, exactly. This right here, I am super excited about Index Explorer simply because of the detail it lets you get into. So, now let's say, you're really trying to figure out how to build out a proper link building program, and you're looking at this going, "All right. We have good links. How do we maintain this? How do we optimize these things?" This is a blueprint for who's linking to you and what they're saying about you. This is an opportunity now for you to contact the websites and say, "Hey, I notice you're using this anchor text to link to me. Instead of click here, can you actually put my product name in there? Here's a sentence I've written for you that incorporates that. All you have to do is copy and paste it in place for me. Thank you very much." If people are wondering how you tracked it on the website . . .

Rand: How much do you have to pay these people to have them do that for you?

Duane: Oh, dude, that's between you and them, right? Personally, I don't pay anybody anything. My own website? If you're going to do it, I love you for it, and I will show you the love later on. If you're going to ask for a handout for it, it's not really how it works.

Rand: Right, yeah. I was going to say, and Bing has been pretty good about penalizing a lot of the links that look manipulative on the Web too.

Duane: Yeah. It's a natural part of keeping things clean, right? At Bing, we are very keen on having a quality driven index. So, the main focus we have is making sure that everything that gets in is a good resource, when someone makes a query they get a realistic answer that is actually an answer to their query. Not, here's some shallow depth data. I'm going to click on it, and then oh, it's not really what I want. I go back and I try it again. We're trying to shorten that number of searches to get to the final answer.

Rand: So, a question that a lot of people have around Bing is often, I've launched my site and I've seen maybe some other search engines pick it up, but I haven't yet been crawled as deeply, or I've been crawled but I haven't indexed. Can this help to answer that question of why that's happening?

Duane: Well, what it will do is, over time, first off you've got to get in the index. If you're not in the index, nothing's going to show here. So what you want to do is make use of the submit sitemap feature we have in here. You know what, I was going to do a blog post on this on the Webmaster Blog but I'll just tell Mozzers, you guys are hearing it here first. We have a quality threshold on our sitemaps. When you build a site map for us, we want it to be clean. When you put a URL into our site map, what I don't want to see in there is any URL that's a 404, 302, 301, anything at all. I want the end state URL only.

Rand: You don't want rel=canonicals.

Duane: Only end state URL. That's the only thing I want in a sitemap.xml. We have a very tight threshold on how clean your sitemap needs to be. When people are learning about how to build sitemaps, it's really critical that they understand that this isn't something that you do once and forget about. This is an ongoing maintenance item, and it has a big impact on how Bing views your website. What we want is end state URLs and we want hyper-clean. We want only a couple of percentage points of error.

Rand: The best of the best 200s.

Duane: Right, because if you start showing me 301s in here, rel=canonicals, 404 errors, all of that, I'm going to start distrusting your sitemap and I'm just not going to bother with it anymore. If the way that you're communicating to me that you have new content is to submit the sitemap through the functionality in Webmaster tools, instantly you're submitting me something that I've learned not to trust because its cleanliness is in question. It's very important that people take that seriously. It's not a fire and forget. Don't just go and grab some random tool when you do a quick search and saw, oh, here's a sitemap generator. It will go crawl my site. Blah, there you are. Forget how many links that actually misses. Seriously, be thoughtful when you build your sitemap. We don't want every single page from every single website. We want your best quality pages and content. So, you as a site owner . . .

Rand: Are there content thresholds that I should be thinking about around this too?

Duane: There are probably are, but those are buried so deep in so many different layers and there's a lot of other influences too. I mean, you can have variations and orders of magnitude and still rank well.

Rand: Okay.

Duane: It's not something that I would really say to folks, look you need to lose sleep over that. This, you should be thinking about and investing in.

Rand: So any of these codes, you really need to worry about.

Duane: Right, exactly. I love this idea, Rand, this whole pick your top 200, whatever the number happens to be for you, pick it and run with it. You don't need everything indexed. Pick your best stuff and make sure that's in there. Make sure your quality content is in there, right? Be sure that you look at the site and say, "What's the goal of this page? Is it to monetize ads? Is it to convert somehow? What is the goal of it? Is it optimized properly to do that? If it is, I want that indexed in the search engine ranking well."

Rand: Duane, are you telling me that we need SEOs?

Duane: Funnily enough, it works.

Rand: Along with this, there's another really cool tab and we're not going to go all through it, but I want to talk about the traffic tab because there's cool stuff in there right now. It will show the rankings data for Bing.

Duane: Exactly.

Rand: It shows something called average rank and average impression rank.

Duane: Exactly.

Rand: So per-click rank and impression rank, what are the difference between the two?

Duane: The way it works out is, because of the volumes that a search engine deals with, like millions of things every minute, essentially, what we're really showing you here is we're saying, when someone did query, the average impression -- where you showed -- is this position. It could be that the average works out to 3.2. So you were roughly the third position for all these queries. The next line over will show you what the click-through rate was. The average click-through rate for you may have been 9.8%, which pretty much falls in line with what we kind of all know and understand.

It's a realistic thing, right? Then the next column over is actually going to show you the average position you were in when that click happened. Now, here's where it gets really exciting for an SEO. If you know that you're in third place most of the time and your click-through rate was 10%, we'll say, but the clicks happen when you're in the first position, how does that 10% that you're getting being in the first position potentially compare with other people who may be in the first position? Now, we don't show you the competitive data, but it should get you thinking about this. Is 10% for me in the first position on this query a realistic amount?

Rand: So, I can look at those numbers, and then I can say to myself, "Wait a minute. This doesn't match up with some of my better click-through rates.

Duane: Right, exactly.

Rand: I should be thinking better title . . .

Duane: Meta description, title, all that, exactly.

Rand: URL string, rich snippets.

Duane: Totally. Speaking of rich snippets . . .

Rand: The stuff that Stefan was showing us, and it just rolled out is pretty slick. You've got that nice sort of box, and there's the guy and he's cooking up some piaya in there, whatever he's got. Then there's the title listing and the meta here.

Duane: Yep.

Rand: I mean that looks really slick.

Duane: It is. I don't know what to say, other than if you're thinking about rich snippets, do it. It's very valuable to a search engine, which means it's very valuable to you and your content.

Rand: What's the how of this? Is there stuff in Webmaster tools now or will there be?

Duane: Not yet. Just as we were prepping for the session, we were talking about future looking things. Rich snippets is one of those still kind of floating for me ideas. I have to find the space for it. I have to understand what the value prop is, what the likely cost is to you, as a business owner. It's one thing if somebody has . . . if there's anybody out there who does plug-ins for WordPress and wants to make a rich snippet plug-in, might be useful to people.

Rand: All right. Rich snippet plug-in . . .

Duane: Because, I mean, you think about the plug-ins we have for WordPress already . . .

Rand: Is Joseph watching this?

Duane: I'm guessing there's already one out there, because rich snippets is a concept that's been around for a while now. So my money is on there's already one there.

Rand: The question is, is the format that you're using the same format that I would use when I am submitting rich snippets to Google, or is this slightly different?

Duane: It's a little bit different, but we're essentially looking for the same kind of points, right? We want this meta markup data to be able to help us understand exactly what this piece of content is. In a lot of cases, it's not going to matter because it's a regular content page . . .

Rand: What is this image? How does it related to this page?

Duane: Right, for that kind of thing, that's really important to us, because, yet still, even though it's 2011 and we are technology advanced, although we're still flying the space shuttle today, search engines still can't actually look at a picture and say, "Ah, that is a fat chef with piaya."

Rand: Must be good.

Duane: Right, exactly. Eats well.

Rand: Awesome. Before we wrap up, there are three questions that I know webmasters should be curious about. You may not be able to give us a ton data about what's happening with Yahoo Site Explorer.

Duane: I'm actually going to not quite pull a no comment on this one, but Yahoo Site Explorer is a Yahoo product, and we don't comment on their product pipeline or things they're working on. What I will tell you is, if you're looking for external link data, it's right here. So we have it for you, right? It's a similar product. We power it.

Rand: Is Bing thinking about some way of showing competitive linking, so like I can go see the links to anyone? Or is that not a product that you're interested in at this point?

Duane: I, personally, have a great deal of interest in it. Bubbling that up to what we can actually bring forward, there's a lot of layers to get through on that. Conversations like that, very nascent at this stage, meaning it's on my whiteboard. However, I am influenced by the greater user population. So, as people have feedback, I'm going to be at SMX in a couple of weeks, I'll be at South by Southwest, SES New York, specifically looking for feedback from people.

Rand: Fantastic and in the blog comments.

Duane: Exactly. If we have like . . . you mean I'm supposed to come to the blog. Oh, you didn't tell me that.

Rand: What? You give up SEO and now you don't come and visit anymore. What's up with that?

Duane: No. Seriously, if people do have feedback for it, right, I capture all of that data because that is a tool that I can use with our product planning folks to say, hey, look, we reached x number of thousands of people with this and the percentage that got back to us was like 18% said they want "A"'.

Rand: So, you're telling me if I throw up a little survey on the blog and I ask people what they most want from Bing's Webmaster tools people . . .

Duane: I will buy you a martini.

Rand: All right. It's done. It's going to happen.

Duane: There you go. There we go.

Rand: Then, static rank, you know people . . .

Duane: Yeah, it's pretty cool.

Rand: Well, MSN and then Bing had this for years, this static rank rate, which my understanding is it has some of the logic of page rank, but a bit more advanced in terms of what it analyzes and what it looks at.

Duane: There are a lot of signals that goes into calculating that, as I am sure there are for page rank. Now that I've kind of come inside . . .

Rand: You can see those.

Duane: . . . I get it now.

Rand: So static rank, we sort of talk about like, hey, Bing wants more webmasters paying attention. They want more people downloading and installing their toolbar. What do you think about putting static rank in a toolbar or in Webmaster tools?

Duane: Personally, I see a great deal of value for people in it. There's even the entire conversation about gaming and all that. We're kind of at the stage now where we're kind of really getting beyond all those things. Like the ability to deal with people trying to game something has evolved immensely. It's . . .

Rand: Showing them a little one through twenty radiated bar isn't going to make them more . . .

Duane: From the company's perspective, however, there's a lot of personal or I should say proprietary investment in this concept. So, what would make it out is got to go through a lot of layers again. However, in your poll . . .

Rand: Put in static rank.

Duane: Why not?

Rand: All right. It's in there.

Duane: I mean, seriously, if there's any tools people think they might want, let us know. I'd be surprised if we didn't already have a version of it floating around the ecosystem somewhere. Part of my job is to bring that all to one spot that's useful.

Rand: I was going to say, you've done some cool stuff to bring Index Explorer into the public facing tools.

Duane: Exactly.

Rand: Last question, and this is around less Bing focused stuff or tool focused stuff, but we have this big content farms update, the farmer update that's coming out from Google, and there's been this discussion ongoing for a while around, hey what's up with people like Mahalo and eHow and Answers.com, who are essentially producing a page for every search term they can possibly ever find. Sometimes of okay quality, other times of really low quality. What's your opinion of how, if you were in charge of the algorithm of Bing and how it got handled, what would you do?

Duane: I'd be looking, and this is the toughest part of it, is separating the wheat from the chaff. These guys seriously have some depth in areas that are legitimate answers to questions. How do you, you can't look at a domain level solution and say, okay, if you're this domain, there you go. It's incredibly hard to crack it. I'd be seriously looking at talking with these businesses and making sure they understand the value and what their role is in the ecosystem.

Rand: I was going to say, could the same thing that you were talking about with sitemaps apply here, where it's like, hey, you guys don't do a good job of curating the content you produce. You're trying to game things. We don't like that.

Duane: I'm working, right now, on my whiteboard, it's this size and it's absolutely full, chock full with content that I have to write, I have to produce. If people who were actually producing shallow and non-relevant content read the basic, the top five points of it and just adhered to that, they wouldn't have an issue. This is my biggest takeaway. Having run the SEO side of something and then come over to Webmaster and I'm kind of looking at it from another point of view now, is seriously sweat your content, sweat being an authority. That's what you have to invest in. If you think there's a short cut to get there, I can almost guarantee you there's not. It takes hard work.

Rand: And it's weird too because I think that a lot of these businesses look at what the algorithms are doing and they say, "But I'm clearly getting away with it. Why wouldn't I take advantage of this in the short term?" Well, maybe you can bring it back to your team. But I would love to hear maybe some content from you guys in the blog in the future like, look, we gave you another 90 days to get your crap out of our index before we do it for you and you're not going to like what we do.

Duane: That's a legitimate approach too. It's our index. We've got to keep it clean and keep the users happy.

Rand: Duane, this has been fantastic. I think folks have likely learned a lot. We're going to grab some screen shots from some of the things that we talked about, put them in here. We're going to run a poll next week and ask people for some suggestions on Bing.

Duane: Awesome.

Rand: We hope that you'll come back and join us again.

Duane: Absolutely.

Rand: Thank you so much.

Duane: Great. Thanks for having me.

Rand: Take care everybody.

Duane: Thanks Mozzers.

Video transcription by SpeechPad.com