Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

  1. Home
  2. SEO Tactics
  3. Technical SEO
Log in to post

Category: Technical SEO

Discuss site health, structure, and other technical SEO strategies.

  • KimCalvert


    I'm seeing about 450 warnings on this. What is "Using rel=canonical suggests to search engines which URL should be seen as canonical." And what do I need to do to fix it?

    | KimCalvert
    0
  • Nobody1533077082756


    Currently working on a project for a medium sized site (http://sleeponcall.com/) but the SEOMoz crawl crawled over 14,000 pages because the report is showing more than 8,000 redirects. The client has no clue how this happened as their previous web programmers may not have been on the ball. What could be causing the problem and what is the best way to untangle this mess?

    | Nobody1533077082756
    0
  • ChristopherM


    Hi folks, I'm working on a website that runs on WordPress and was not updated by the owner, this has resulted in a malware injection and now when you search the companies name in Google, the site links appear with words like Viagra, et al. I've seen this a number of times, so I went through the code and have removed all the malware. I presume I now have to wait for Google to recrawl the website and update the site links? Is there anything else I should be doing to speed up the process? Thank you 🙂

    | ChristopherM
    0
  • lzhao


    We'll soon be launching a redesign, on a new platform, migrating millions of pages to new URLs. How can I tell Google (and other crawlers) to temporarily (a day or two) ignore my site?  We're hoping to buy ourselves a small bit of time to verify redirects and live functionality before allowing Google to crawl and index the new architecture. GWT's recommendation is to 503 all pages - including robots.txt, but that also makes the site invisible to real site visitors, resulting in significant business loss.  Bad answer. I've heard some recommendations to disallow all user agents in robots.txt.   Any answer that puts the millions of pages we already have indexed at risk is also a bad answer. Thanks

    | lzhao
    0
  • EwanFisher


    Hey guys, I'm currently working on cleaning up our link profile and have been looking at several tools. Has any one used this from http://www.linkresearchtools.com do you think its worth investing in? Matthew

    | EwanFisher
    0
  • JonasNielsen


    Hi all,We're running a used car search engine (http://autouncle.dk/en/) in Denmark, Sweden and soon Germany. The site works in a conventional search engine way with a search form and pages of search results (car adverts).The nature of car searching entails that the same advert exists on a large number of different urls (because of the many different search criteria and pagination). From my understanding this is problematic because Google will penalize the site for having duplicated content. Since the order of search results is mixed, I assume SEOmoz cannot always identify almost identical pages so the problem is perhaps bigger than what SEOmoz can tell us. In your opinion, what is the best strategy to solve this? We currently use a very simple canonical solution.For the record, besides collecting car adverts AutoUncle provide a lot of value to our large user base (including valuations on all cars) . We're not just another leech adword site. In fact, we don't have a single banner.Thanks in advance!

    | JonasNielsen
    0
  • bonmaklad


    Ok my VPS is in America and all my websites with .co.nz have been found by google but luggagenz.com doesn't come up... I then realised that all my websites are classed as american... umm how do I change this without moving my vps?

    | bonmaklad
    0
  • Benj25


    Our homepage on our ecommerce site has a category as part of the front page. Unfortunately the top of the category says: Items 1 to 9 of 22484 total Is there a way through GWMT or any other method of stopping Google from adding this on to the front of our Meta Description? Or is the only solution to remove it from the page all together?

    | Benj25
    0
  • MPO


    Hi Seomoz, I have been strugling to get www.texaspoker.dk out of what seems to be a keyword specific penalty (we are on page 5 on "poker" and "online poker"). First I thought it was Penguin related, but I'm not so sure any longer. I have removed all the bad links to my site possible (it's not easy to get other people to remove links, I can tell you), and I have reported all the links that I would like google to "ignore" (reconsideration request) ... all in all I have requested for reconsideration 5 times, and - despite some small changes - I got the same answer every time. We violate the quality guide lines and we should be looking for unnatural links pointing to our site. If any one - by having a look at our site - have any idea what could be wrong, please don't hold back, we would love to hear your point of view. Right now we are in the middle of making our partners take off their site wide links to us (the partners you'll find if you click the flags to the right in the top of the home page). On Texaspoker.dk we only link to the partners from the home page, but maybe we should consider to take of even these links? No one is really clicking on them anyway. Another thing, which is only under consideration, is to ask our partners from Betxpert.com (with whom we have exchanged our news feed - you will find their feed if you schroll down the home page) to set the feed to "no follow" and do the same our selves. What do you think of this thought? As far as I can see, there is nothing wrong with the on page optimization, but maybe some one can see what I don't see? Again - what ever thoughts you guys may have - shoot ... i'm ready to take all bulets 🙂 Thanks in advance! Nicolai, Texaspoker.dk

    | MPO
    0
  • richcowley


    Our UE team has presented me with a new site structure where the content (folders) does not match the hierarchical directory structure (in the CME). I.E Sub-sectors, sectors and product pages are ALL just 1 directory off the root. example.com/sector example.com/sub-sector example.com/productpage FYI 'normal' folder hierarchy would be; example.com/sector/ example.com/sector/sub-sector example.com/sector/sub-sector/productpage I cannot find any SEO disadvantages re; crawl, if anything the SE's will crawl more efficeitly with clearly less depth... higher 'deep content', and a better nav - which is technically a sound solution with link consistency throughout  - 1 to 2 clicks to all pages. Only disadvantage might be a user confusion... which can be off-set with contextual breadcrumbs. Are there any PURE SEO disadvantages to a structure this illogical? Note - This does not abuse any Search Engine guidelines. Thanks for reading, Rich

    | richcowley
    0
  • conversiontactics


    Hello, On May 19 , 2012 Google webmaster sent notification "Google Webmaster Tools notice of detected unnatural links to " both sites haven't  lost any ranking or traffic as yet. I am worried, Should i panic,what should i be doing , will it be going down anytime soon? how to naturally build links?

    | conversiontactics
    0
  • NaescentAdam


    Hi, I'm running Joomla 1.7 with SEF on and I'm trying to do a htaccess redirect which fails. I have approximately 100 in effect so far and all working fine, but I have one snag. Index.php is not working as I need it to when it's redirected to www.myurl.com/ If I turn on index.php redirect to root using this code #index.php to root
    RewriteCond %{HTTP_HOST} ^myurl.com$ [OR]
    RewriteCond %{HTTP_HOST} ^www.myurl.com$
    RewriteRule ^index.php$ "http://www.myurl.com/" [R=301,L] And then go to www.myurl.com/test.html I'm redirected to the homepage. I think this is because all pages are index.php in joomla. SEOMOZ and Google both think that index.php and root are duplicate pages. Does anyone have any advice for overcoming this? Thanks, Adam

    | NaescentAdam
    0
  • philscott2006


    Hi guys, I'm a novice really when it comes to SEO, yet have taken it in house for the next year or so, firstly because I have had my fingers burnt twice...and secondly, to allow me to recoup some of the loss from my prior campaigns. One thing I have noticed on my site (which uses a Shopify E-commerce CMS), is that Shopify duplicates a url for each my products. An example of this is http://www.vidahomes.co.uk/collections/designer-radiators-heating/products/reina-aliano
    http://www.vidahomes.co.uk/products/reina-aliano Both products provide exactly the same information, yet appear in different ways subject to how the customer finds them. I contacted Shopify to find a fix to this issue when I noticed a high amount of Duplicate Title Tags in my SEO crawl. Their response was as follows. Using a rel canonical link will help prevent duplicate content issues with search engines. All you need to do is add this line of code: **<link rel="canonical" href="{{ canonical_url }}" />** ** before the tag in the theme.liquid file. It’s that simple :)** The theme liquid file basically generates the outer template for the whole site, and is only compromised when over-ruled. This all seems a little too easy for me, so I am hoping whether someone can elaborate as to whether this will work or not, as I'm not entirely sold on their response. I was always under the impression with canonical tags, that they should be added to the header section of the duplicate page in question, which refers back to the original page. The code I have been told to add above implies that the canonical tag would be added to every page in my site so the Google robot would have a hard time in finding anything at all of relevance Thanks in advance for any assistance with this. Kind Regards Phil Scott Vida Homes

    | philscott2006
    0
  • HughesDigital


    I have a client that uses Sharepoint as their CMS and they are having a tough time getting cannonical tags to work on the website without breaking other items. Has anyone had a similar experience?  Do you have any resources that could help? Any input would be greatly appreciated.

    | HughesDigital
    0
  • stevetemple


    Hi there just joined had nightmere of a time trying to get a website up and running..... now i have 2 .... one marketing person did and one i did the one i did performing better on google but other onre looks more profetional is there a way i can conbine the 2 under one site..... the one that looks better and getting the benifit of the one thats performing better...... Thanks steve......

    | stevetemple
    0
  • RaksG


    Hi, I have a question wrt Mobile version of a site. Previously, we had a mobile site which is no longer active and there are possibilities of resurrecting it in future, so we have a 302 redirect which points to the homepage (desktop version). Currently, the mobile site is indexed by the search engines. To avoid the duplicate content issue, is it recommended to use robots.txt and block the spiders from mobile content or apply 301 redirect until the mobile site is up and running,  OR continue with the 302 redirect. Any suggestions will be helpful. Thanks,

    | RaksG
    0
  • sherohass


    Is it better to keep the Transaction/ Payment pages on a commercial website as the only secure ones (https) and remainder of website as http? Or is it better to have all the commercial website as secure (https)?

    | sherohass
    0
  • BetAmerica


    Just got the new crawl report, and I have a recurring issue that comes back around every month or so, which is that a bunch of pages are reported as duplicate content for themselves. Literally the same URL: http://awesomewidgetworld.com/promotions.shtml is reporting that http://awesomewidgetworld.com/promotions.shtml is both a duplicate title, and duplicate content.  Well, I would hope so! It's the same URL! Is this a crawl error? Is it a site error? Has anyone seen this before? Do I need to give more information? P.S. awesomewidgetworld is not the actual site name.

    | BetAmerica
    0
  • chansen


    We often get asked by clients if they have a lot of content about a particular subject: would it be better to break that information out into smaller chunks as separate pages, OR would it be good to build a tabbed content container or accordion feature on one single page? Does anyone have any opinion on this in regards to SEO?

    | chansen
    0
  • semer


    "GET /favicon.ico HTTP/1.0" 404 "GET /Scripts/AC_RunActiveContent.js HTTP/1.0" 404 I checked the server log and found the 2 files ( favicon.ico and AC_RunActiveContent.js) always return 404 error code, I do not have 2 fileson the server, how can I fix the problem.

    | semer
    0
  • SanketPatel


    Hi every one I have 2 websites with same domain with targeting different country  name for example www.domain.com.au and the other is wwww.domain.co.nz .  So my question is that in webmaster tool my one domain doesn't shows back links data or search query data. why this is happen?

    | SanketPatel
    0
  • Nightwing


    Hi from sunny 19 deg C wetherby UK 🙂 I read somewhere some links cannont be passed thru if they use Javascript. With that in mind on this site  http://www.collegeofphlebology.com/ theres a horizontal scrolling nav bar illustrted here: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/footer-nav-indexable.jpg Ive got two questions please: 1. Are these links ok in that they do not represent an indexing roadblock
    2. How can you tell if an internal link cannot be passed Thanks in advance 🙂

    | Nightwing
    0
  • irvingw


    I added the rel="author" tags to the blog posts last week and updated the authors page with a link to the Google+ account, but I have yet to see the authors photo surface in the Google Results. Example URL: http://spotlight.vitals.com/2011/10/dr-richelle-cooper-testifies-against-dr-conrad-murray-in-trial/ Can anyone identify what else needs to be done?

    | irvingw
    0
  • ben1000


    A number of sites are copying - either 100% word for word, paragraphs, or sentences of my content and are ranking higher. Some sites are doing this with permission/properly and are linking back to my article Others are not linking back or giving credit. Some of these sites, in some cases are ranking higher than me in Google results. What can I do?

    | ben1000
    0
  • CFSSEO


    The issue: text on a page available only  by pushing a button or clicking Read More and the text slides up via AJAX. Will Google penalize a site doing that via hand edit?  I think that according to Matt Cutts, as long as the text is available to the user then it's ok.   I want to double check on this.

    | CFSSEO
    0
  • ChristopherM


    Hi folks, I've got a fairly interesting scenario. I'm trying to rank this page (http://www.staysa.co.za/sa/1-2-0-0-1/East-London/accommodation) better for the term, "accommodation east london". The client isn't keen on making many changes and it was built horribly with ASP, half CMS, half not. I have made the following changes today: I introduced two paragraphs of text below the H1 tag. I changed "East London Bed and Breakfast", "East London Conference Venues", "East London Cottage / Chalet" to just "Bed and Breakfast", "Conference Venues", "Cottage / Chalet" as the continual key phrase duplication in my experience is a bad move. I've made a change to the title tag (this is a huge mission as it's not CMS controlled, so I had to teach myself some basic ASP to do so). Meta data.. nightmare to change unfortunately, at least not without rewriting part of the CMS. I'm wondering, are there any other on-site factors that I'm missing? I'm not a fan of site-wide links, so I don't want to put an exact match anchor text link from the sidebar/footer to the page, not unless someone can motivate why I should. Keen to hear everyone's opinions 🙂

    | ChristopherM
    0
  • jamesm5i


    Howdy mozzers I have a car dealership site with an inventory page that allows users to specify dealer location, model, make, year, etc. with drop-down menus. Each new choice changes the URL parameters and unspecified categories appear as "...&model=&year=..." I've attended to the URL parameters in Webmaster Tools, and according to Google's new URL parameters video, it looks like they've got a handle on navigating the site. That said, I do get warnings in my seomoz crawls that these URLs are overly dynamic (there are up to 12 parameters per URL). How concerned should I be? How would the Moz community suggest handling this many parameters in my URLs? Thanks as always

    | jamesm5i
    0
  • Prime85


    Hello, I created a wordpress blog a while back, http://plastic-bins.com/ If you go into one of the pages on the blog (for example: http://plastic-bins.com/plastic-shelf-bins/ ) you will notice a link after the text/content telling you where you can purchase the stuff that is talked about on that page. There is one link back to the e-commerce site on every category page on the blog. Does anyone know if these links will help me or hurt me in terms of ranking in the SERPS? Thanks

    | Prime85
    0
  • RGEQUIPNET


    SEOmoz's Crawl Diagnostics is complaining about a duplicate page error. The example of links that has duplicate page content error are http://www.equipnet.com/misc-spare-motors-and-pumps_listid_348855 http://www.equipnet.com/misc-spare-motors-and-pumps_listid_348852 These are not duplicate pages. There are some values that are different on both pages like listing # , equipnet tag # , price. I am not sure how do highlight the different things the two page has like the "Equipment Tag # and listing #". Do they resolve if i use some style attribute to highlight such values on page? Please help me with this as i am not really sure why seo is thinking that both pages have same content. Thanks !!!

    | RGEQUIPNET
    0
  • Sparkstone


    I was under the impression that when a search engine comes across duplicate content it won't be able to determine which one is the original. Is this not the case?

    | Sparkstone
    0
  • -Al-


    Hello to all, I have a number of Linking Domains on our Open Site Explorer Report that no longer exist. I've run URL checks on just a sample of the list, and found that approx. 35% of that sample are from now dead Linking Domains. Can someone help? If these Linking Domains are defunct, how can I remove these? Does Google reflect negatively on these dead Linking Domains in our SERPs? Has anyone experienced this before? What action did you take?

    | -Al-
    0
  • Seiyav


    I'm thinking of using unbounce.com to create some content driven pages. Unbounce is simple, easy-to-use, and very easy for non-devs at my company to create variations on pages. I know they allow adding meta descriptions, title tags, etc and allow it to be indexable by Google, but I was wondering if there were any potential downsides to using unbounce as opposed to hosting it myself. Any help would be appreciated!

    | Seiyav
    0
  • TinaGammon


    I read somewhere a while back that having a dedicated ip address was helpful for seo if this true or just another rumor? Also I read you should purchase your domain name for multiple yrs, what do you guys think?

    | TinaGammon
    0
  • Johnny4B


    To the SEO experts, this may well seem a silly question, so I apologies in advance as I try not to ask questions that I probably know the answer for already, but clarity is my goal I have numerous sites ,as standard practice, through the .htaccess I will always set up non www to www, and redirect the index page to www.mysite.com. All straight forward, have never questioned this practice, always been advised its the ebst practice to avoid duplicate content. Now, today, I was looking at a CMS service for a customer for their website, the website is already built and its a static website, so the CMS integration was going to mean a full rewrite of the website. Speaking to a friend on another forum, he told me about a service called simple CMS, had a look, looks perfect for the customer ... Went to set it up on the clients site and here is the problem. For the CMS software to work, it MUST access the index page, because my index page is redirected to www.mysite.com , it wont work as it cant find the index page (obviously) I questioned this with the software company, they inform me that it must access the index page, I have explained that it wont be able to and why (cause I have my index page redirected to avoid duplicate content) To my astonishment, the person there told me that duplicate content is  a huge no no with Google (that's not the astonishing part) but its not relevant to the index and non index page of a website. This goes against everything I thought I knew ... The person also reassured me that they have worked within the SEO area for 10 years. As I am a subscriber to SEO MOZ and no one here has anything to gain but offering advice, is this true ? Will it not be an issue for duplicate content to show both a index page and non index page ?, will search engines not view this as duplicate content ? Or is this SEO expert talking bull, which I suspect, but cannot be sure. Any advice would be greatly appreciated, it would make my life a lot easier for the customer to use this CMS software, but I would do it at the risk of tarnishing the work they and I have done on their ranking status Many thanks in advance John

    | Johnny4B
    0
  • ChrisInColorado


    I have two customers that sell the same product.  Their content is logically similar because they are selling nearly the same thing.  The part numbers, descriptions, code base and underlying structure are different however. One customer is complaining that because I host both of these sites in the same server cluster on the same subnet I am negatively affecting their ranking. To what extent might this be true?  I've always held that unless the content is identical the subnet thing was just a myth. Any opinions?

    | ChrisInColorado
    0
  • semer


    I want to Block or prevent pages being accessed or indexed by googlebot. Please tell me if googlebot will NOT Access any URL that begins with my domain name, followed by a question mark,followed by any string by using Robots.txt below. Sample URL http://mydomain.com/?example User-agent: Googlebot Disallow: /?

    | semer
    0
  • semer


    I want to use robots.txt to prevent googlebot access the specific folder on the server, Please tell me if the syntax below is correct User-Agent: Googlebot Disallow: /folder/ I want to use robots.txt to prevent google image index the images of  my website  , Please tell me if the syntax below is correct User-agent: Googlebot-Image Disallow: /

    | semer
    0
  • Prime85


    Good Morning, We have a Magento shopping cart based site running on RedHat version of Linux. We had a CMS block created for the homepage of http://goo.gl/JgK1e designed  to be visible only on the homepage only and nowhere else. We copied the entire site structure onto a new URL http://goo.gl/XUH3f . (this one running on CentOS) and have an odd situation on our hands... Even though the CMS block “static_after_footer_block” is “enabled”, it either completely disappears (moments later), or whenever it does display, it is visible in ALL levels of the site (not just the homepage it was designed for) Other than this anomaly, the site seems to be operating correctly… Anyone out there with some insight? Thanks!

    | Prime85
    0
  • SavilleWolf


    Hi all, I'm new to SEOmoz and SEO in general really. I run a small but well regarded freelance website and graphic design business, and until very recently had an employee who handled the SEO side of things. I'm now looking to step into this role myself and hopefully learn the in's and out's of SEO. I've no doubt there will be much to learn, but the SEOmoz tools and it's community seem excellent and helpful. My question then is basically, if WordPress generated HTML code can have an effect on SEO, when it's reported as invalid by tools such as the W3C HTML validator? I'm used to hand coding the majority of my websites for clients, where creating valid HTML and CSS code is something I can do with relative ease. A new client however wants to use WordPress - for ease of updating the site content themselves. The client does however consider any potential SEO implications to be a very important factor in choosing a hand coded vs. WordPress based website. I am aware that WordPress itself is just a means of generating HTML code, and that to the search engines there is no difference between this and the hand coded websites I usually produce. However if WordPress is generating HTML that is being reported as invalid, would this make the search engines penalise the site? On a second note, will the search engines look negatively on a WordPress site where it is being used as a standard website, and the content may not be updated as frequently, as say, a blog? Thanks for your time, and I look forward to hearing your suggestions.

    | SavilleWolf
    0
  • webfeatseo


    I have a client who had a screwy url structure based off of parameters and all. They hired a developer that added the keyword to the end of the url and set up 302 redirects to the new keyword included url. Since then the entire site has virtually gone missing in the results but it is not penalized. I put in a request with webmaster tools for reconsideration and they said there was no penalty. I only just found the 302 problem today and think this is probably the problem. Could this remove a site from the search results?

    | webfeatseo
    0
  • semer


    Crawl Results show there is 404 error page which is index.htmk **it is under my root, ** http://mydomain.com/index.htmk I have checked my index page on the server and my index page is index.HTML instead of index.HTMK. Please help me to fix it

    | semer
    0
  • AndreVanKets


    We are currently in the process of creating new pages for keywords very similar to our main landing pages. Example: Page 1.) "Family Holiday in {Location}" Page 2.)  "Family Vacation in {Location}" Page 3.) "Family get-away in {location}" All 3 keywords are currently optimised for the 1st page "Family Holidays in {Location}" but was wondering if there is any benefit or if its worth creating sub pages for these alternative keywords. I personally don't think we should do this as Google will know to rank our main page for secondary/synonym keywords via on page seo and links with anchor text, but can you foresee any negative aspects if we went ahead and created these children pages being linked to from the top level keyword's page? Or do we continue optimising our main landing page for related/synonym keywords? Thanks in Advance GREG

    | AndreVanKets
    0
  • PGD2011


    For some reason, I'm getting extra words in my title tags. For example, I wrote "Washing Machine Widgets | Acme Widgets, Inc. | Acme Widgets Inc. Anyone have any idea why I'm getting the extra " | Acme Widgets Inc."? Thanks!

    | PGD2011
    0
  • danatanseo


    We have several old domains. One is http://www.ccisound.com  - Our "real" site is http://www.ccisolutions.com  The 301 redirect from the old domain to the new domain works. However, the 301-redirects for interior pages, like: http://www.ccisolund.com/StoreFront/category/cd-duplicators  do not work. This URL should redirect to http://www.ccisolutions.com/StoreFront/category/cd-duplicators  but as you can see it does not. Our IT director supplied me with this code from the HT Access file in hopes that someone can help point us in the right direction and suggest how we might fix the problem: RewriteCond%{HTTP_HOST} ccisound.com$ [NC] RewriteRule^(.*)$ http://www.ccisolutions.com/$1 [R=301,L] Any ideas on why the 301 redirect isn't happening? Thanks all!

    | danatanseo
    0
  • newstd100


    Apologies - not strictly a search question, although I am concerned that it may be negatively impacting my sites. If I enter an invalid URL - the "page not found 404 error" doesn't seem to be working. I can enter http://www.%sitename%/index.php/randomnamemadeup/ and it displays the homepage (albeit with a broken CSS). I can't fathom it out - any help appreciated. Thanks, David

    | newstd100
    0
  • schof


    We're considering changing the URL's for our blog posts and dropping the date information. Ex. http://spreecommerce.com/blog/2012/07/27/spree-1-1-3-released/ changes to http://spreecommerce.com/blog/spree-1-1-3-released/ Based on what I've learned here the new URL is better for SEO but since these pages already exist do we risk a minor loss of Google juice with 301 redirects?  We have a sitemap for the blog posts so I imagine this wouldn't be too hard for Google to learn the new ones.

    | schof
    0
  • jamesm5i


    Howdy mozzers I'm hoping you can lend some advice. I'm dealing with a site now with loads of URL parameters. It's a vehicle dealership group which hosts its entire inventory from multiple locations on one page, sorted by parameters. Example inventory URL: www.dealership.com/car-inventory.asp?pa=&ns=10&so=m&sor=DESC&ma=&mod=&mt=&yr=&bs=&pr=&t=used&ln= Where pa (page no.); ns (number of vehicles shown); so (sort by condition); sor (sort order); ma (make); mod (model); yr (year); bs (body style); pr (price range); t (type - new, used, etc.); ln (location no.). As you can imagine this generates a gazillion URLs (or slightly less). Any thoughts on best canonicalization options? Thanks as always

    | jamesm5i
    0
  • tpt.com


    We launched our new site 2 days ago , since site was down for 12 hours for maintenance, we saw google webmaster tool shows this error . Since then google hasnt crawled, its been 36 hours. Do we need to do anyting? We have close to a million page google crawled before and I am wondering if this will effect anything.

    | tpt.com
    0
  • linklater


    Hi, Do you know if there's any way to track how often robots.txt file has been crawled? I know we can check when is the latest downloaded from webmaster tool, but I actually want to know if they download every time crawlers visit any page on the site (e.g. hundreds of thousands of times every day), or less. thanks...

    | linklater
    0
  • sherohass


    Is it true that https pages are not crawled and indexed by Google and other search engines as well as http pages?

    | sherohass
    0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


Start my free trial


Browse Questions

Explore more categories

  • Moz Tools

    Chat with the community about the Moz tools.

  • SEO Tactics

    Discuss the SEO process with fellow marketers

  • Community

    Discuss industry events, jobs, and news!

  • Support

    Connect on product support and feature requests.

Looks like your connection to Moz was lost, please wait while we try to reconnect.