This post was written by James Svoboda of Realicity Search Marketing in Eden Prairie, Minnesota.

Monday, April 20th, 2009: Today I celebrate the completion of my first decade in search.  I have been waiting for this day with some trepidation for the past 6 months or so.  I am not really sure why or even what this anniversary really means.  Does it mean that I am some sort of expert?  Well, anything is possible.  Or does it mean that I have wasted the past decade with little to show in an industry that my friends and family can hardly even understand?  Hmm, I hope not.  In truth, it means I not only have a job that I like, but a career that I love.

I represent one of a hundred or maybe even a thousand tenured SEO professionals in the world who have spent their time doing and not blogging about doing.  The only notoriety that we typically gain is the word of mouth referrals from our happy clients.  To some of us, these clients have become more like friends.  These friends make what we do just that much more rewarding.  I cannot even imagine doing anything else and came to the realization several years ago that I am going to spend the rest of my life with a career centered around the web.  I guess only time will tell what that Tuesday morning in 1999 really started.

History & Change

I had a great history teacher in the 7th grade who regularly told us that “we need to learn from our past so we do not repeat our mistakes in the future.”  I am sure he was not the first, or the last, to use a saying like this, but remembering it makes me wonder if someday public schools will teach students about the history of the internet along with the European Renaissance and Industrial Revolution.  And since you can now earn a degree in internet marketing from several accredited universities and colleges like Rasmussen or Full Sail, it might only be a matter of time before someone like Rand or Danny starts teaching a course on the History of SEO at U DUB.

In the past few months I have been doing a lot of thinking about the changes that have occurred on the web since I started.  I managed to get some of them down in a list and looking it over reminds me of so much, probably in the same way as cracking open an old high school yearbook or visiting with an old lifelong friend who you have not seen in years.  Maybe it will invoke some old memories and you can reminisce too.

Search Engines

AltaVista had been the Google of that era.  There were other somewhat popular search portals like Lycos, HotBot, DirectHit, Northern Light, Excite, and of course Yahoo, AOL and MSN had their loyal following.  I guess AltaVista’s popularity is why Overture/Yahoo eventually bought it.  I just don’t know why they ruined it.

I can even remember when we would resubmit each of our client’s pages to MSN every month because their submission page fed into Inktomi (which provided the search results for MSN, Yahoo, HotBot, Lycos and a few others) and you would receive a rankings boost for “New Pages.”  Inktomi was great for us for a while.  They eventually came out with a Paid Inclusion that you could purchase for a yearly inclusion for each page.  No more resubmitting each month, yea!  You received such a ranking and traffic boost that we started purchasing inclusion for all of our clients without charging them because it produced such better results that made us look great, and our clients loved us for it.  That program was great until Yahoo bought Inktomi and changed the paid inclusion program from a yearly fee to its current a yearly fee plus a PPC fee called Search Submit.  We pretty much had to drop all of our clients from the program at the time, which resulted in all of the pages that paid for inclusion getting dropped from their index (and along with them traffic, rankings and sales).

In those days the search engines were spending tons on TV commercials in order to capture a larger search share.  There were the Lycos Dog commercials, AltaVista had Pamela Anderson, the Excite ads were kind of edgy, and Yahoo had some good ones like the Comb Over and Raise the Dead.  However, my all time favorites were the HotBot Investment Tips and Political Scandals commercials in what I call the “Old Links” series. I think I found this series particularly funny because I am an SEO and can remember when you could do a search and within the first 3 pages find a listing or two that would lead to a 404 page.  Also, the distinguished gentlemen they used were great!  The staff at Search Engine Land also put together a search engine commercial montage that you should check out.

Eventually, many of the original search engines failed to turn a profit and were slowly being purchased for their search market share, mostly by Yahoo, CMGI or InterActiveCorp.  The Searching Graveyard also has a few interesting tidbits on some oldies like Magellan, Deja and OpenText.

Meta Search

DogPile, MetaCrawler and WebCrawler were the three that I was first introduced to, and in fact all are still online today. What I really loved about Meta Search Engines was not their ability to pull results from a dozen different engines at the same time, but just the fact that their entire premise was based on how bad each individual search engine’s results could be.  Even search results for top keywords were hit and miss, and quite often we were forced to jump from one engine to another in order to find relevancy.  I think this was primarily due to the fact that algorithms of the day were heavily based on on-page factors like keyword density, and links were not a major ranking factor in most of the popular engines.

Web Directories

Wow, have these changed!  We used to make getting listed in the Yahoo Directory and the Open Directory - DMOZ a priority because of the traffic they generated.  Now any actual visitors that originate from them are probably either a link builder, a competitor or an editor.

Disney-owned Go.com, originally called Infoseek, launched a directory called the GoGuides that was pretty popular.  You could join to become an editor and then list your clients that were sorted alphabetically and ranked by 1, 2 or 3 stars.  When Disney shut down this directory, a few of the editors spun off a directory of their own called, of course, GoGuides.  GoGuides.org is still online today with pretty much the same format that they brought with them from the original Go.com directory.

At one point Looksmart.com was a valuable directory that powered some results for MSN, Inktomi and several others.  You could initially pay a $199 review fee for a permanent lifetime listing.  This was something that we only did sometimes for our clients because the traffic coming out of Looksmart was not always enough to warrant the one time fee.  They eventually changed their ad model to a Pay Per Click format and at the same time informed all of their previous customers who had purchased a lifetime listing that they were converting all of them to the new PPC program.  They then funded your account with your original submission fee and you were charged a $0.15 per visitor fee until your account reached zero and you were de-listed.  A class action law suit was brought against them and I believe they settled out of court.  MSN was their big distribution partner at the beginning of their PPC/directory end.  Once they lost MSN their stock dropped, their directory was eventually taken down, and they were never really the same again.

The original Snap.com portal also built a web directory that was of brief significance and provided good traffic.  This one was probably around the shortest amount of time of any of the Dot Com directories.

Pay Per Click

The first PPC search engine that I managed a campaign for was GoTo.com.  They were simple, with a “Visible” bidding process that could let you bid based on current bids without a quality score algorithm to consider.  Originally you could only bid on each keyword by itself, or what is today called Exact Match.  You always knew which keywords your traffic was coming from.  The only downside for some is that you had to do some really deep and creative keyword research to reach your PPC potential.  The GoTo search term suggestion tool (keyword research) was the best and most reliable keyword tool I have ever come across.  You could punch in any term and know that you were going to get 50 results that were a derivative of your words.  No synonyms or related terms to confuse you and make you wonder what keyword you actually just searched for.  If you were given less than 50 results on the page, then you had reached the end of that search and it was time to try a different version.  They only showed you results that had 25 or more searches from the previous month, and if you were given any results that did not end at or close to 25, you had to keep searching for additional derivatives.  It was also very easy to find more versions as each keyword result was linked so you could click on it and run another search. 

The GoTo tool only had 2 little drawbacks.  The first was that it combined singular and plural versions of most keywords except a few like link & links, service & services, company & companies, and a few others.  At one point I think I had a list of about a dozen versions that would need to be checked individually.  The second was that it was not an intuitive tool to use if you were researching keywords for a campaign about a topic that you did not really know anything about, like a few software and business consulting clients I worked with early on.  I owe much of my keyword research skillz to GoTo.  Thanks.

At one point GoTo.com sued Go.com for infringement based on Go’s logo looking too similar to theirs.  Go had a green stop light logo and GoTo had the full green, red and yellow light.  GoTo won a bunch of money and made some big headlines as the then little engine that sued Disney and won.  Eventually GoTo changed their name to Overture and then Yahoo bought them up and changed their name again by rebranding them as Yahoo Search Marketing.  All of those name and logo changes must have really pissed off the legal staff at Disney that lost the infringement case!

There once was another little PPC engine that started out by providing ads for About.com.  It was called Sprinks.  That current version of Sprinks did not last very long, as Google soon bought them up in order to compete with Yahoo/Overture.  They were quickly absorbed into Google and either stripped of its PPC technology or rebranded as Google AdWords.

There were also several other little PPC engines that tried to make it, like FindWhat - now Miva, Bay9, Kanoodle, 7Search.com, and I even remember one called 3Apes.com.  Some of these are still around, some have been rebranded or bought by a more profitable company, and some have died and gone to PPC heaven.  I hear it is nice there, with absolutely no click fraud to speak of.  Of all these, only 2 have so completely rubbed me the wrong way that today I have to check myself before I getting into a profanity laced tirade when I hear their names.  Bay9.com was a company that shorted an affiliate payment on a site that I placed their ads on.  Their rep was great and said all the nice things and then never would answer or return calls again.  This might be one of the many reasons why Bay9 is no longer in business.  FindWhat (now Miva) is the other one.  They were the first PPC engine where I could pinpoint click fraud, call my rep, get the run around about adding funds back into my account from the “bad” partners account, and have it happen again and again.  That is another one that refused to return my calls after a while.

Before GoTo there was... I am not sure, as I can’t recall their name now.  During my first year I spent a good amount of time doing keyword research based on GoTo.com keyword traffic stats.  I remember my first introduction to GoTo when I was told their short history.  It was also during this conversation that I remember hearing about a PPC predecessor to GoTo.  Not a large one, but a predecessor nonetheless.  I do not remember much about this site/engine/platform, so if you have any additional bytes of knowledge, feel free to share them.  I only really remember that they did not last long.  They were not perceived very well because they were charging for something that people thought should be free, and at that time the internet was free and nobody should cage it or have to pay for it!

Things were much different back then.  The DOT COM had not yet bubbled or burst, banner ads were the main way to pay for search traffic, and I am pretty sure most people still thought the World Wide Web was flat and could only be accessed through a phone line.

Oh, and MSN eventually created their own PPC platform called adCenter.

Affiliate Programs

The first affiliate program to generate real buzz online was the Amazon.com associate program.  They had a product to match just about every website out there.  If you were a dog breeder, you could link to Amazon pages about books on dog training, breeding, or just about anything else you could think of.  I remember seeing Amazon’s founder, Jeff Bezos, on the Tonight Show when he explained how he originally boxed books for shipment while kneeling on the floor.  His story was about investing in how they did things like buying tables so they did not have to kneel any longer.  Up until that point Amazon had yet to turn a profit, and he did not shy away from talking about how they were continually reinvesting profits back into the company.  At the time I kept thinking that they were either going to crash so hard that many of the Amazon staff would end up jumping out of windows, or they were going to become an amazing success story.  I was actually very happy for them when they reported their first quarter of profit.

After Amazon came Commission Junction.  CJ was an affiliate marketplace that, like Amazon, had just about something for every website.  If you were a dog breeder, you could probably find a few pet supply stores to link to and make some commissions.  The great thing about them was that their affiliates paid much better than Amazon.  The biggest drawback that I saw with Commission Junction was the fees that they charged for being the broker were pretty high.  This eventually led to lower commission rates for the affiliates as the web stores and CJ tried to maintain their revenue.  I am not sure where they stand in today’s affiliate world because I left Commission Junction as soon as Google launched AdSense and have had no reason to return.

Google AdSense has not changed much over the years.  They are still easy for publishers to use, pay very well for each visitor, and they still have very few ad formats for being such an advertising powerhouse.

UGC & Social

At times I have a tendency to group these two together, mostly because in the early days websites that allowed User Generated Content or User Edited Content (UEC) also included some form of a social experience and community.  Back then directories were my UGC & social experience.  I was an editor at DMOZ and both versions of GoGuides.   I added and edited directory content, read the newsletters and the editor forums, and made an effort to better my “community.”  There were several Webmaster and SEO forums, but I did not spend too much time in them as they seemed to be filled with hacks that passed misinformation.  I was never interested in chat or IM, and I don’t even know what ICQ stands for.  (Well, now I do.)

I remember when About.com first became popular.  They had just changed their name from The Mining Co. and there was a buss around it in a similar fashion to Wikipedia when it first hit the scene.  I once checked out the available topics section and then decided not to try and join.  I just didn’t see how I could find enough time to develop that much content.  I guess About didn’t see how they could either, and that is why they opened it up to Guides.

CitySearch has been around for a while and I once used them religiously, but they are now a sinking ship.  Yelp discovered the true value in UGC that CitySearch had discarded in favor of advertising revenue: free business listings.  It's a simple platform for reviews that is open to almost anyone, and they encourage offline community involvement and have user profiles with photos for recognition, community status, list building and bookmarking, review tracking and a way for people to connect with each other.  They really have thought of just about everything that CitySearch chose not to.

Analytics & Reporting

Webtrends log file analyzer was the first program that I ever used for client reports.  Its reports were similar to some of the free solutions around, like Webalizer and AWStats.  It was great for identifying the search engines and keywords that we were generating traffic from, and it also provided a little bit of data like browsers and operating systems. 

We eventually gave in to client requests for ranking reports and started using WebPosition Gold.  We knew that traffic was what really mattered, but we found by providing these reports that our clients rarely questioned campaign progress and traffic numbers.  I think it was because they had a little easier time understanding a top 10 position as a measurement as compared to a report with hundreds of different keywords and visits.  WebPosition is now owned by Webtrends and has remained mostly unchanged for years.

Google Analytics became a mainstay of client reports.  Its reporting features, conversion tracking and visual elements are great, especially for the price!  I still run a ranking report once a month just to save time of checking top keywords myself and to help watch for movement on whole keyword segments.  Like almost all of Google’s non-search technology, GA was once part of another company called Urchin and was absorbed into the Google collective.

Algorithms & Updates

AltaVista’s Black Monday happened only 6 short months after I started in SEO.  I came into the office on what seemed like an ordinary Monday to what eerily felt like the first day of the stock market crash that led to the Great Depression.  Most of our clients' traffic had disappeared from our biggest source at the time.  We normally would check a few client log files to see how traffic went over the weekend, but that day we checked all of them.  There was a tiny hint or feeling of doom in the room.  We were almost panicked.  What would our clients do if we could not figure this out soon?  Would they leave us?  What about company revenue?  Had it tanked enough to affect me?  Eventually it all worked out, but what an ordeal!

Google’s Florida was another tough one.  It happened right before the main holidays.  It really screwed most of our clients, many of whom had ecommerce sites that really needed to sell over the holiday season.  Florida took longer to overcome than AltaVista’s Black Monday and not just from a ranking and traffic standpoint, but from the loss of clients as well.

It was not long after Florida that Yahoo decided to change their Paid Inclusion program from the yearly fee to the yearly fee plus cost per visit.  I guess since Google’s updates were given names in alphabetical order just like hurricanes, it is only fitting that we were hit hard by those two in succession and felt like we living through hurricane season.

Now that I have spent a little time looking back, pondering search, my career and how the internet has evolved, I can’t help but wonder as to the changes that are coming up and in particular what’s next for search.  Maybe in another 10 years or so I will write a follow up and take an even longer stroll down search memory lane.  I plan on being around then.  SEOmoz will probably still be here.  Hopefully you will too.

Cheers!