For search startups the real barrier to entry is the index

I was drawing up a brief document for my CEO yesterday around search, the developing technologies and the startups in the space. What caught my attention was the number of starts calling themselves "Google-Killers". It took me a while to put my finger on it but I just didn't agree no matter how cool/ground breaking/esoteric their technology was.

Why don't I see these companies such as Powerset or Hakia as "google-killers"? Index. The simple fact is that unless the company has an index that is significant percentage of the Google or Yahoo or Microsoft search index's they can't compete. Early on the size of the web was such that a new search engine could easily develop a useable index. Now it is orders of magnitude harder to not only develop the index but also make it fast, reliable and generally useful.

There is a way to mitigate this barrier to entry and that is to pick a vertical and index that. Indexing a vertical is a much easier that trying to index usable portion of the internet. Given the nature of the technologies that Powerset, Hakia et al are deploying I think health (which is also relatively open and new) would be good fit for them.

[I had included Yedda as a Google-Killer but as Yaniv Golan pointed out in the comments, Yedda is along the lines of Yahoo Answers with knowledge ranking.]

Tags: Search, Google, Startups, Powerset, Yedda, Hakia

A human experience

I have found it fascinating to watch the every widening ripples that the Kathy Sierra et al saga has unleashed.  Tom O'reilly has just posted the basics of a code of conduct, which continues the expanding ripples.



Even more fascinating is how entirely human the whole issue is.  How entirely human the responses and counter responses, the alliances and friendships are.  All in all, I found it demonstrates one very, very important fact: geeks, techs are as irrational and human as the rest of the population.  They are as likely to get angry, to be mean as anyone.



What concerns me most is the seeming wide acceptance of meaness not only in Tech based blogs but also in political blogs as well, even across the whole blogsphere.  The acceptance of meaness hidden behind freedom of speech. Does this not de-value the concept and importance of freedom of speech when it becomes something to prop up an individuals own pettiness and anger? If I remember my history lessons correctly, freedom of speech is about disagreeing with government.  It was never there to protect some one from the consequences of every thing they say.  Something a long history of court decisions has upheld.



The web and internet has grown since the early days.  The norms that arose in those days hung together as a majority of the users belong to the same community and with that community came limits on behaviour.  Now that the majority of users do not come from the same community the norms of behaviour to a greater or less extent do not have the weight or power they use to.  The internet has descended into a "Lord of the Flies".