This Is How Search Gets Commoditized

The first Google logo, when the project was called BackRub and focused on Internet “backlinks.” Lore has it the hand is co-founder Larry Page’s.

(original post at Searchblog)

Once upon a time when search was new, Google came along and put the whole darn Internet in RAM. This was an astonishing (and expensive) feat of engineering at the time — one that gave Google a significant competitive moat. Twenty years ago, very few companies had the know how or the resources to keep an up-to-date copy of the entire web in expensive, super fast silicon. Google’s ability to do so allowed it unprecedented flexibility and speed in its product, and that product won the search crown, building a trillion-dollar market cap along the way.

Since then compute, storage, and engineering costs have declined in a kind of reverse version of Moore’s Law. Pretty much anyone with a bit of funding and some basic Internet crawling skills can stand up a web index — but there’s been no reason to do so. For 15 or so years one of the biggest clichés in venture circles was “no one will ever fund another search engine.” (A second cliché? “No one’s ever said “Just Bing it.”)

Google had won, and that was that.

All of this came back to me when reading an eye-opening leaked memo over on Dylan Patel’s SemiAnalysis newsletter. Titled “We Have No Moat, and Neither Does OpenAI” and written by a senior researcher at Google, the memo is a devastating critique of Google’s approach to competing in the world of generative AI. SemiAnalysis placed a strong caveat at the top of the text, and I’ve not been able to find confirmation of the memo’s claims (Bloomberg has more here). But the essence of the memo’s argument resonates: Both Google and OpenAI are going to lose out in the race to dominate the AI future because they are refusing to play in the domain of open source software, where legions of developers are racing ahead of companies who are trying to go it on their own.

But while the “open vs. closed” debate is fundamental to the future of the Internet (and our society), that’s not the only thing I’m on about today. What I realized today was something more concrete and astonishing. With GPT technologies, pretty much every startup can now put a decent facsimile of Google in RAM, and according to the memo, they can do it for pretty much peanuts.

Put another way, Google’s been commoditized.