LLMs, from buzz to reality
November 30, 2022. OpenAI launched ChatGPT, finally making technology that had been a decade in the making available to a broader consumer audience. ChatGPT ended up being the fastest product ever to reach 100 million users (January 2023, 2 months!), and in the process OpenAI reached some $2 billion in ARR by December 2023.
As consumers got excited about a conversational interface that works (yes, not those clueless customer support chatbots we’ve all been using), businesses had to come to terms with the fact that LLMs were here to stay. As a result, companies of all sizes and industries began to work with LLMs, including many that wouldn't be considered "early adopters" according to Rogers' “Diffusion of Innovations”.
Fast forward to today, and we’re reading about LLMs going into production left and right. “Head of AI” positions have tripled since 2018, and by the end of the year around 80% of Fortune 2000 companies will have dedicated AI leadership. It takes 18-24 months for the majority of companies to adopt new technologies. Recent announcements of LLMs in production are the first batch of “Early Adopters”, as per Rogers’ classification. November 2024 will mark 2 years since the launch of ChatGPT, and we are seeing signs of a much wider "Early Majority" adoption towards the end of the year.
What’s next? LLMs user intelligence.
Every day, millions of users interact with LLMs in various ways.
What’s next? Common sense suggests making sure these users have a good experience. Is the LLM really helping users to solve their problems?
How many prompts does the average user need to get there?
And by the way, what do most users chat about?
If that sounds like a no brainer, that’s because it is.
Learning from existing users behaviour is the fastest way to improve your product over time and deliver a ROI for the business. But if you’re still skeptical, let’s rewind the clock to the late ‘90s and look at what happened to website analytics.
Learnings from Web Analytics
The first web page ever went live on August 6th, 1991. It was dedicated to information on the World Wide Web project and was made by Tim Berners-Lee. By 1995, the number of websites had reached 23.5k and a small startup from San Diego, Urchin, made a simple bet: every company with a website needs to know what’s going on that website in order to make it better.
According to Urchin’s founder Scott Crosby: “Honda.com was unable to process each day’s Apache access log before the end of the current day, dooming them to fall ever more behind.” Of course, the same logic applies to LLMs. If anything, this is even more true for LLMs than it is for websites, because users don’t just click buttons, they write (unstructured) text.
If your LLM users generate more interactions than your AI/Product/Customer Success/Marketing team can read - let alone digest - in a day, how are you ever going to improve the LLM user experience?
So what happened to that Urchin startup?
Chances are you're still using their product. After successfully riding the Internet wave, they were acquired by Google in 2005 and rebranded as “Google Analytics”. By that time, there were 65 million websites, i.e. 2,000x more than when Urchin got started.
It’s fair to wonder where we would be without Urchin. My two cents is that we would have less user-friendly, uglier websites and, as a result, we would not be using websites as much.
LLM user intelligence
If the history of website analytics is any guide, here is what will happen with LLMs:
1. In the the next 3-5 years, most companies will spin up their own conversational AI interfaces, much like every company has its own website.
2. User intelligence/analytics for LLMs will be essential for companies to stay ahead of the curve and continue to deliver a best-in-class experience to users and a good ROI to investors.
If you are already working with LLMs and thinking how user intelligence can be applied at your organization please book a meeting with us to learn more.
