🔮 E06: Large Language Models and the Assembly Line
By 2030 LLMs will have increased labour productivity by 20% in OECD countries versus 2022 benchmark levels.
Started a little blog just to get some traffic. How do we all feel about a hip hop differentiated deep tech newsletter? Ben of a16 did the hard thing about hard things. I had more feedback on Eminem and Nate Dogg than on neuromorphic computing. Sad.
But the law of audience capture begs me to do more.
There are three rap quotes in this newsletter. Bring me the quotes and some fair criticism in the comments and you get a prize I guess? You want money? You want street cred? I tell you what, I’ll go out and buy the CD and ship it to you. Old school. Who said newsletters were boring? Tell you friends.
Why bother reading on?
How LLMs finish the job the Internet started in the digital supply chain
Why LLMs are the only technology that scored 5+ in our assessments
Why LLMs will rescue crypto (Yes, it does need rescuing!)
Onward Into Battle, Lawrence
📬 Mailbag
On neuromorphic, Thanks to
for the shout out. “Lawrence Lundy and Lunar Ventures built a research tool to understand the deep tech landscape, and they’re making it freely available. “
I made it ma.
Some feedback from John Dent:
“I am sorry, but your statement about what is available, is factually incorrect. BrainChip does have a commercially available product, and it has been available for months. Their "Akida" chip technology is evolving and the second generation chip is soon to be released. You might want to review your investment strategy. Their tech is certainly not OLD school.”
Fair play. More here. The press release says: “Introduces Vision Transformers and Spatial-Temporal Convolution for radically fast, hyper-efficient and secure Edge AIoT products, untethered from the cloud”
This hints at some early thinking about value proposition: edge, low-latency, secure, and local. They also say “Akida’s neuromorphic processing platform is event-based, fully digital, portable, and proven in silicon.” This pitch validates the assessment that digital will be more popular over analog short-term.
I spent some more time this week on neuromorphic and analog computing. I’ll explain/try to explain the difference next week. What they both have in common is a massive change on the demand-side….
✍️ The Model T Moment for Bits: LLMs and the Software Assembly Line
LLMs are moving too fast and no-one can accurately reflect the state of the art succinctly. This is my attempt to take a “helicopter” view so to speak. More than anything, having conducted nearly 100 assessments now, very few, i think 5 technologies score a 5+ in terms of impact. (Maybe it should be a 5++ because it offers a frisson of existential risk). No other technology scored 5s across the board. So it might seem like the next hype cycle, but it is materially different in terms of drivers, novelty, restraints, and timing than metaverse or web3.
Every pundit worth their salt has a view on AI and LLMs. The best take imo is this.
“PT2030 can “work” and “think” quickly: I estimate it will be 5x as fast as humans as measured by words processed per minute [range: 0.5x-20x], and that this could be increased to 125x by paying 5x more per FLOP”
Many people far smarter than me have more informed opinions. Marc Andreessen also has opinions, lol (too soon?) You didn't create the first web browser or a $35bn venture fund, etc, etc. Your simple words just don't move me. I don’t have a good view on AGI. My small contribution is one of framing. We see lots of market maps, and stacks, and flow charts, but they aren’t useful as a tool for thinking. I find the production, distribution and consumption framing to be the most useful mental models for LLMs and the emerging value chain.
Production: Volumes increase, costs decline. LLMs are a digital mass production technology. The best analog is the mass production system and the introduction of the assembly line by Ford in 1913. At first at the low-quality, high volume end. And as the models get better, increasingly higher-quality. Nothing is ever zero-marginal costs, as software production gets cheaper, the bottleneck will be hardware (Elad at Lunar calls this “AI Austerity”),and energy. Energy is always the bottleneck eventually, this is why energy abundance should be the narrative not energy transition.
Distribution: The Internet delivers a near zero-marginal cost of distribution hence Cloud and SaaS overt the past 20 years. Relative to physical mass production with transport and logistics, distribution costs are extremely low. This is important because it means agents can interact with other agents/digital systems at low cost (BabyAGI, AutoGPT) creating long and complex interactions. Mass manufacturing lowered production costs but not distribution costs (although they were reduced materially by road and rail networks and then latter the shipping container). The Internet decreased distribution costs, but production was still manual and human. This is why we haven’t seen the expected productivity gains. LLMs reduce production costs with no distribution bottleneck.
Consumption: Number of relevant humans hasn't increased materially. Attention span hasn't increased materially. If the story of the last 10 years has been an abundance of content, expect 100x more. The spoils to Ben Thompson’s Aggregators will be 10x. Owning the demand-side is about to get 100x more valuable. LLM-created content will have higher engagement and lower churn and LLM-created content with have no royalties so higher margins. The spoils to brand and trust will increase.
Sidebar: The debate over more AI-generated content to lower costs versus less AI-generated content because of the harms will be the dominant left versus right cleavage in our politics this decade. If I am on the right, I will argue we should let AI-tutors and AI-doctors compete with teachers and doctors to reduce costs. The deflationary impact will be vast. But If I am on the left, unemployment, lower human wages, and potential harms and crimes will need to be addressed. If I were a politician, here’s what I would say: costs are rising. we need to reduce them. we have new tools (AI) to make things (bits) for cheaper. lets invest and encourage more of these tools. costs will fall. you will be better off and have more money in your pocket.
short summary
A large language model is an AI model trained on large amounts of data to generate human-like output. GPT-4 is an example of a model that can perform various language-related tasks, such as translation, summarization, and text generation.
long summary
Large Language Models (LLMs) are machine learning algorithms that learn statistical associations between billions of words and phrases to produce human-like natural language outputs. Large Language Models (LLM) describe models typically tens of gigabytes in size with billions of parameters and trained on large amounts of data, often multiple petabytes. LLMs entered the public consciousness in 2020 with the release of OpenAI’s GPT-3 and 2022 with ChatGPT, which materially pushed forward the state-of-the-art in creating human-like text. The past three years has seen unprecedented progress in extending LLMs into other content generation tasks such as DALL-E2 for text-to-image generation, ProGen2 for predicting protein structures, PaLM-E for robotics, and most recently multimodal LLMs like ImageBind which processes images, text, audio, depth, thermal, and IMU (Inertial Measurement Unit) data.
viability: how technically mature is the technology? (5)
Language models have evolved from basic N-gram models to more complex recurrent (RNN) and long short-term memory (LSTM) neural networks. In 2017, Google introduced a new network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. This technique paved the way for whole sentence and paragraph processing and parallelisation. Since 2017, language models have got progressively larger from Elmo in 2019 with 1 billion parameters through to GPT4 with 1+ trillion parameters in 2023. New models differentiate through larger context windows (100k tokens) with Anthropic’s Claude; compression (QLoRA) as with Guanaco-65B; or WizardLM for complex instructions. On the R&D side, the transformer model will likely have to be replaced/tweaked as the attention layers do suffer from a quadratic dependence on its input. This is a good survey on so-called efficient transformers.
drivers: how powerful are adoption forces? (5)
Predominately a supply-side driver. Language models, mainly n-gram models, have been a part of daily life for over a decade through search engines, voice assistants and transcription services. These services were satisfactory, but transformers from 2017 brought a higher accuracy above and beyond the human level. The performance of GPT-3 and DALL-E-2, in particular, broke through to the mainstream, bringing attention and investment. There has always been a large demand for natural language interfaces and computers that can understanding language. But it really has been only since 2020 when the capabilities were good enough to serve this demand.
novelty: how much better relative to alternatives? (5)
LLMs are highly novel. They compete with alternative, older techniques like n-grams and long short-term memory (LSTM) networks, but the performance is so much better that they aren’t competitive. LLMs are also novel in their generality; the same model can be used for question-answering, document summarisation, text generation, etc. The extent of this generality is an open question. Some argue LLMs can scale and with enough parameters can solve all the problems (aka AGI). Others argue that other algorithms will always be superior for other types of tasks like strategy or locomotion. The most likely outcome is a system that incorporates different algorithms like reinforcement learning or evolutionary algorithms with LLMs to most efficiently solve a broader array of tasks. The upcoming DeepMind chatbot Gemini will likely showcase this sort of system.
diffusion: how easily will it be adopted? (5)
Extremely powerful tailwinds because of the generality of the technology and the sheer number of applications. Regarding market access, at multiple billions and soon trillions of parameters, training cost and available training data are the main bottlenecks. Very few organisations have the resources to train LLMs and can limit and slow access as they wish, as seen by the limited roll-out of GPT-3/4 and DALLE2. However, the AI open-source community is strong, with EleutherAI, HuggingFace, and StabilityAI, among others forcing faster roll-outs. In terms of performance improvements, cost is likely to become the rate-limiting factor. But cost is never a long-term headwind when demand in strong. Expect the AI goldrush to bring new entrants to the hardware market limiting rent seeking and predatory pricing. Short-term we will see more AI ASICs and FPGAs, and more investment in photonic, analog, and neuromorphic chips pulling forward timelines.
impact: how much value is created? (5+) High certainty
high-impact scenario (mass) The speculative highest impact scenario sees LLMs scaling to multi-trillions of parameters and as one of two pathways to artificial general intelligence (AGI) (The other in whole brain emulation). (Assuming a pathway exists at all) This scaling hypothesis is just that, a hypothesis, but even at some low probability (<5%) of success, AGI is probably the single most impactful technology humans will create as it will solve complex problems around energy, space exploration, longevity, etc (or kill us all). Slightly less impactful is a much more probable scenario in which LLMs are the digital equivalent of the factory system innovation. Mass production and lean production revolutionised physical production; LLMs are the first digital production tool enabling digital production to scale. The productivity dividend from the digital revolution and Internet has so far failed to materialise. This failure is because we lacked a mass digital production system. LLMs could be the tool to deliver the expected productivity gains. This scenario sees all content creation disrupted in the next decade as humans and LLMs combine to automate and augment all digital production.
low(ish)-impact scenario (niche) The low to medium impact case with a very low probability as of June 2023 is that LLMs are close to peaking in terms of performance due to available data and/or power consumption. You have to argue that existing hardware, namely electronic von-neumann designs are simply too power hungry to support materially larger models. This case would see a stagnation in progress until new lower-power hardware such as analog, neuromorphic or photonic chips are fabricated at scale. But considering the demand for LLMs, it’s more likely demand will pull forward the supply of new chips even faster avoiding a stagnation.
timing: when will the market be large enough or growing fast enough for risk capital? (2020-2025) High certainty
This is a no brainer. Arguably no technology has seen faster development and progress in the last three years, from 1 billion parameters to 500 billion and likely over 1 trillion in the next 12 months. Size is a reasonable proxy for performance, so we will, in the next 2/3 years, even better models applied to a broader range of application areas. ChatGPT grew it's customer base to 100 million in less than two months, the fastest adoption of a software product ever. We are beginning to see an explosion of commercialisation across a wide range of applications. There is a strong case to be made that the LLMs will see the fastest adoption of any tool in human history.
what are some of the most important companies?
Aren’t all the startups, AI startups now? I’m not sure how valuable a short list of startups are but fwiw, here’s three I am excited about.
Anthropic (if you use ChatGPT, try Claude, it has a longer context window)
InflectionAI (they do https://heypi.com/, a personal LLM. Inflection AI is an ‘AI Studio’ specializing in creating personal AIs. It was founded in early 2022 by Mustafa Suleyman (co-founder DeepMind), Karén Simonyan (ex-DeepMind), and Reid Hoffman. And news alert: now with 4bil!
Langchain, combine LLMs and build apps. This is where the sophisticated chain-of-reasoning agents will come from.
Overrated or underrated
Correctly rated. I originally had this as “underrated” but surely in June 2023 everybody has internalised the importance of the technology. I do think however that progress will not scale linearly. We will see continued fast progress in next 3 years, followed by bottlenecks due to lack of data and cost of energy, until new non-electronic hardware arrives from 2027+. This is the bull case for neuromorphic and analog chips, too.
what are the main open questions?
X-Risk: Will scaling LLMs create AGI? And Is the creation of AGI an existential threat to humanity?
general versus specific: Many vertical-specific LLMs like BloombergGPT or Nvidia BioNeMo claim to perform better than general models for specific industry tasks. Will larger models erode the benefits of finetuning over time? Thick servers and thin clients? Or thin servers and thick clients?
There's numerous ways you can choose to earn funds: How will the LLM value chain develop? The demand and competition make it likely that foundational models will be open-source (unless the US determines otherwise for national security purposes). What is the right analogy: cloud and SaaS, or android/AOSP and mobile apps?
And I know my rights, so you gon' need a warrant for that: Lot’s of companies are worried about their data and queries being used as training data. What is the right equilibria here? On-prem local LLM deployments? Or maybe some federated learning+privacy-preserving solution? Has anyone put a LLM in a TEE yet?
When will LLMs use money? Inevitably this will be crypto. With unstoppable crypto networks how do we think about liability in this regime? How do we think about private/secret agents using zcash or FHE chains? How will we know if an agent or human controls a crypto wallet?
Who benefits? If LLMs are as impactful as most people think, do we need a new tax regime for the beneficiaries? Or a new type of company like the nonprofit and an capped-profit model a la OpenAI. Why are none of the new AI startups following this path? What role could crypto play? What does a DAO-owned LLM look like? What does crypto network of data assets, models and agents look like where value is distributed to data producers? (Nudge: Nevermined)
2030 prediction
By 2030 large language models will have increased labour productivity by 20% in OECD countries versus 2022 benchmark levels.
