It's never easy to admit to being dead wrong, but in this instance, it would be doing the SEO community a great disservice not to make my mistakes public.

In a thread at SEOChat - More on Term Weight..., I abuse members of the notion that keyword density can be used to calculate the importance of a term or phrase in a given document. I propose that, much as the IR community does, we instead use term weight to conduct this kind of analysis and help to better 'optimize' our keyword use in documents. That suggestion, as it turns out, won't work either.

My friend, Dr. Garcia, who many know as Orion from SEW, posted a few great follow-ups to the thread explaining why term weight calculation of a single document probably won't help you create a better optimized document for a global search engine. His reasoning is simple - search engines use term vectors, which are calculated across the millions of documents they index, to form ideas about context and term use. Therefore, the importance of a single keyword term or phrase in a document cannot be measured without access to this additional data. It's also impossible to measure a document without conducting linearization, which one can do through automated software.

I was worried that there would be no alternative method for "optimizing", but thankfully, Orion has some ideas as to how to achieve a scientific method of keyword use in a document. It's called on-topic analysis and he's posted about it in the past at SEW and has a good article on the subject at Miislita (his site).

On-topic analysis can't be done entirely by machine (unless you're a search engine). It requires human involvement to help categorize and sort topics from broad to narrow. It also uses Dr. Garcia's infamous c-index calculations to measure the connections between terms in a document to their use across the www. I'm in the process of ordering up a report from him today and will post what he allows me to share so we can all learn a little more about a real way to optimize keyword usage for SEO.

NOTE: To learn more about term vectors and the "whys" behind this approach, Dr. Garcia recommends a book called Information Retrieval: Algorithms and Heuristics, which he's reviewed here.