What do search engines know?

I just checked out a keyword phrase on MSN search that Travel Lists has always ranked high on in google & yahoo (in google it has not moved more than one place in two years - a useful benchmark) and couldn't find Travel Lists anywhere on the first five/six pages. Stopped looking after that.

The relevant page in Travel Lists matches the keyword phrase EXACTLY - it is a comprehensive answer to the search being made. However the results in MSN are junk. Some good and rather obvious sites languish on pages three and four, and sitting in pole position on results page number one.... is my pet hate.

I'm not going to name the site (which is why I'm not detailing the keyword phrase) because I don't want to slag them off in public. But it is another travel directory, and one which 'punches above its true weight' in SERPs (Search Engine Results Pages) on all the search engines. It carries quite a lot of destination content - which is not particularly unique - but very little on travel providers. Often listings of travel providers for a particular destination are either non-existent (blank page) or, as in this case, are limited to two companies who sponsor the page. Particularly irritating when Travel Lists lists 22 specialist operators from the UK and 23 local travel providers.

It always winds me up when I see this site high in the SERPS for a destination or subject on which it has no information. I won't publish a list of travel providers until I am satisfied it covers all or nearly all the key players. If Travel Lists has any shortcoming it is that it doesn't cover enough destinations (it's ok on holiday types). There are plenty of obvious missing countries in the directory, but that's because there is a huge quantity of pages 'under construction' with already sizeable listings... but still not good enough to publish yet.

There are those who think that web directories are being made obsolete by increasingly clever search engines... and you can understand how they look at the development of the semantic web and the sophisticated ways in which google et al are processing and sorting web pages, and see that happening.

But then, thankfully, you are brought back to earth with a bump. In this case MSN is producing rubbish SERPs in response to a search for travel providers, but none of the major search engines are good, or even close to good. If a searcher uses a search engine (on its regional setting) to find a list of travel providers for a particular destination they will get appallingly bad results: just a few specialists (ironically some of the most useful ones will be paid-for listings), some travel giants, and hopefully a couple of proper directories like Travel Lists, Travel Quest, or Tuttinsieme to get the searcher back on track!

"...and the travel associations' directories!" somebody will be thinking now.

Yeah, and them too. The ABTA, ANTOR, AITO, ABTOF directories, which are all excellent and comprehensive directories.... but,caveat, of their members only.