🔮 E13: Chiplets - How Chip Lego is Driving AI Progress
Yo, chipsets be monolithic, inflexible and gigantic. Chiplets break it down, make it small, modular and automatic
Every week, I do a State of the Future Briefing Note on one important technology, from Vector Databases, LLMs, and Decentralised AI to Optical, Neuromorphic, and Analog Computing. If you’re new, join the World’s best investors and subscribe.
The semiconductor industry is enormous. Enormous and slow. Take chiplets, a modular approach to building computer chips by combining multiple smaller “chiplets” instead of a single monolithic design. The first chip with a chiplet architecture was the Virtex-7 FPGA by Xilinx, now AMD, in 2011.
In 2021, 10 years later, the market was worth $3 billion, accounting for a tiny 0.5% of the total semiconductor market at $600 billion. The chiplet market is growing at like 40% a year and will be worth roughly $20 billion in 2027, so it might make up 5% of a trillion-dollar market. That’s 16 years after the first chip was commercialised. And that’s ignoring the years of R&D before the commercial launch. The idea of the disaggregation of the integrated circuit design into smaller physical pieces of silicon had been in R&D at Intel and TMSC for a few years before, at least. So for clarity, let’s say peak to trough, 12 years to $3bn and 20 to $20bn. Not good venture capital numbers. And chiplets are *just* a packaging innovation. We aren’t talking about new transistors using light or qubits here.
I mention this because timing. I’m (and most VCs) are looking for a discontinuous break in a market. A change or catalyst that changes the market structure materially. Disruption, not incremental change. Optical chips. Quantum. Analog computing. Neuromorphic. Disruptive computing paradigms reshaping markets. But if chiplets took 12 years to get to $3bn, what number do we put on technologies that need a new supply chain? 15 years from lab demonstration? These are big numbers. A VC fund is 10 years. Timelines are different for hardware. Deeptech isn’t just hard work, it’s long work, too.
But obviously the starting point matters. A chiplet startup in 2015 is very different from a chiplet startup in 2023. So here we all are. Why should you care about chiplets?
Well this from my colleague Claude over at Anthropic sums it up:
Yo, chipsets be monolithic, inflexible and gigantic
Chiplets break it down, make it small, modular and automatic
Interconnect, stack 'em high, leverage that modular logic
Die-sized building blocks, snapping together like LEGO bricks
Modular chip floorplan, scalable for the win
Mix and match capabilities, customize within
Heterogeneous integration, optimal for each section
Advanced packaging, next generation chip invention
Chiplets breaking limits of the monolithic ways
Flexibility, customization, performance and speed pays
The future looking bright for this modular chip revolution
Chiplets bringing options, driving chip evolution
Smaller dies better yield, cost goes down, value gains
Packaging multiple chiplets, next level for the brains
So all my designers, chip heads, listen up
Scope this new architecture, chiplets in your cup
The future is modular, no more monolithic (producers note: 5% market share by 2027)
Chiplets bring the diversity, added benefits automatic
That’s actually *pretty* good right? I mean it’s no Lupe or Andre 3000, but it’s better than Lil Durk for sure. Anyway…
Onwards. Let Us Face The Future Together. Lawrence xx
🧩 Chiplet TLDR
Summary: Modular approach to building computer chips by combining multiple smaller “chiplets” instead of a single monolithic design
Viability: 4 - Transitioned from research to early commercial adoption in HPC, AI, networking markets, although work on interconnects and EDA needed for wider applicability
Novelty: 4 - Modular integration provides advantages over both monolithic SoCs and multi-chip packages by enabling heterogeneous integration at the package scale.
Drivers: 5 - Demand for customisation, HBM for AI accelerators and the need for higher manufacturing yields along with maturing supplier ecosystem makes drivers highly favourable.
Diffusion: 4 - Design complexity, lack of standards, and ecosystem maturity create short-term moderate headwinds against adoption.
Impact: 3 - An incremental technology delivering efficiencies but more importantly powerful customization features, in particular enabling hybrid chips.
Timing: 2020-2025 - $3bn market growing fast at 40% to $20bn by 2027 driven by importance for AI and HPC accelerators.
2030 Prediction: Captures 8% of semiconductor architecture market share
Opportunities: (1) Photonic interconnects, (2) ML-based EDA tools, (3) Mixed-signal/hybrid chiplet-architectures (optoelectronics, analog-digital, quantum-classical)
Summary
Chiplets are an emerging approach to designing and manufacturing semiconductors where large, complex chips are assembled from smaller modular blocks called chiplets. This modular architecture allows chip designers to mix and match chiplets with different functions like processors, memory, sensors, etc. to create customized chips.
Chiplets promise several benefits over traditional monolithic chip designs. Using smaller chiplets improves manufacturing yield and allows the use of diverse process nodes optimized for different chip functions. Chiplets also enable reuse of pre-designed and pre-tested chiplets across multiple chip products, reducing costs. Assembling chiplets allows for the creation large chips comparable to monolithic designs but with lower costs and risks. It also enables heterogeneous integration of different process nodes and technologies like analog, digital, RF, MEMS, etc. on a single chip.
The chiplet approach is analogous to board-level integration and packaging of multiple chips onto a printed circuit board but at a finer scale. The two main integration approaches are 2D side-by-side mounting of chiplets onto an interposer and 3D stacking of chiplets using through-silicon vias (TSVs). High density interconnects like microbumps enable communication between chiplets.
Viability (score: 4)
Chiplets are currently transitioning from early research to commercial viability. DARPA initiated research on chiplets starting around 2005, which demonstrated the concept. Around 2015, advanced packaging companies like ASE, Amkor began developing interconnect and integration technologies to enable chiplets. Recently, semiconductor companies like Intel and AMD have introduced chiplet-based products like Foveros and Ryzen Threadripper PRO CPUs, while Qualcomm has announced plans for chiplet-based smartphone chips.
Ongoing research is developing finer-pitch interconnects beyond microbumps, TSVs, and interposer technologies to enable high-bandwidth density communication between chiplets. For example, TSMC is working on integrating its Chip-on-Wafer-on-Substrate (CoWoS) packaging with Integrated Fan-Out (InFO) and System-on-Integrated-Chips (SoIC) technologies to enable smaller distances between interconnects, measured in μm (micrometers, millionths of a meter). TSMC aims for 10μm pitch density, compared to 40μm pitch for standard microbumps. DARPA's XlicSiP program has demonstrated interconnect pitches below 5μm using electroless plating. Georgia Tech research has shown silicon nanowires can enable pitches below 0.1μm. Startups like Ayar Labs are developing chip-to-chip optical I/O to provide high bandwidth density. These examples demonstrate the work underway to push interconnect density significantly higher. Heterogeneous integration of beyond-CMOS devices like MEMs, sensors, analog, and RF remains an area of innovation. But chiplets are now sufficiently mature for commercial adoption in HPC, AI, networking, and other markets, evidenced by new chiplet-based product announcements. Overall the technology viability of chiplets is high at 4 out of 5, having passed proof of viability and now seeing early commercial adoption.
Novelty (score: 4)
Chiplets are a packaging technology primarily competing against conventional monolithic chip integration and multi-chip packages. As a mental model, all the computing components are further away on a chiplet than they are on a single piece of silicon. So at a high-level, chiplets trade-off bandwidth for customization.
Conventional SoC Monolithic Chips:
The entire chip is fabricated as one piece of silicon in an integrated process flow. All components like CPU, memory, analog, etc. are built together.
Tight integration enables high performance. But limited to one process technology. On-chip bandwidths can exceed >10 Tb/s.
Chiplets:
Separate chiplets are fabricated using optimized processes. E.g. digital chiplets in a CPU process and analog chiplets in a RF process.
Chiplets are integrated together at the package level using an interposer and dense interconnects like microbumps. But microbumps enable chiplet interconnect bandwidths up to ~1 Tb/s, slower than on-chip bandwidths >10 Tb/s. Although photonic interconnects are likely to increase this number.
Enables heterogeneity and reuse, but with tighter integration vs multi-chip packages.
Multi-Chip Packages:
Chips are fabricated independently, usually on planar (flat 2D) substrate.
The chips are connected on a printed circuit board using solder bumps or wires.
Much looser integration and slower interconnects between chips vs. chiplets.
Chiplets introduce a modular approach to chip design which provides benefits over conventional monolithic integration like improved yield, mixing process nodes, and IP reuse. While not completely novel, chiplet-based modular chip architecture provides a new middle ground between packaging multiple discrete chips on a PCB and monolithic SoC designs. It improves on existing multi-chip packaging by enabling finer-pitch interconnects while retaining the flexibility of composing chips from smaller pre-tested elements. Very importantly, chiplets are necessarily more expensive than monolithic SoCs. Chiplets sacrifice leading edge transistor density but gain cost savings through reuse and yield gains from smaller dies. Additional packaging costs offset some savings. Adoption is being driven more by performance, customization and time-to-market advantages versus outright cost savings thus far. But it is expected that chiplet costs will to reach cost parity as production scales.
The novelty factor is 4 out of 5 because chiplets don’t enable totally new features per se, instead they optimise the design and packaging process delivering efficiency gains.
Drivers (score: 5)
Demand: The main market drivers propelling adoption of chiplets are the need for heterogeneous integration, modularity for customization, and improved yield/cost. Heterogeneous integration allows combining diverse compute (CPU, GPU, AI), memory, analog, RF, sensor functions on a single chip which is not possible with monolithic SoCs. Modularity through chiplets enables product designers to mix-and-match chiplets to create customized chips optimized for target applications in HPC, AI, 5G, autonomous systems etc. This is particularly important for the design of AI accelerators which use high bandwidth memory (HBM). HBM requires a silicon interposer to connect to the main chip due to its high density of through-silicon vias (TSVs). Using a chiplet design allows the HBM and main logic chip to be manufactured separately then integrated onto the interposer, overcoming the difficulty of manufacturing TSVs on a large monolithic chip. Finally, using smaller chiplets improves manufacturing yield. Smaller chiplets have less area, so there is less chance of defects occurring during manufacturing compared to larger die sizes. By combining multiple smaller chiplets with high yields, the overall yield for the complete chip is higher than if manufactured as one large monolithic die.
Supply: All the demand in the world wouldn’t have mattered a decade ago. There have been previous attempts at modular chip architectures like TI’s OMAP chips for mobile devices and Sun Microsystems’ MAJC 5200 processor using modular cores. But earlier interconnects were too low density and performance. But advances in silicon interconnects, 2D and 3D packaging now enable assembling chiplets into complex chips. For silicon interconnects, microbumps, copper pillars, and silicon interposers allow fine pitch connectivity of <10μm between chiplets, improving bandwidth density vs wire bonding or laminate substrates. 2D packaging technologies like TSMC's Integrated Fan-Out (InFO) and Intel's embedded multi-die interconnect bridge (EMIB) use silicon bridges and fine pitch interconnects to horizontally integrate chiplets on an interposer or substrate. And 3D packaging technologies like die/wafer stacking approaches like TSMC's Chip-on-Wafer-on-Substrate (CoWoS), Integrated Fan-Out on Substrate (InFO-oS) integrate chiplets vertically using through-silicon vias (TSVs).
The technology and manufacturing ecosystem is maturing to commercialize chiplets. With strong demand drivers and an improving supply ecosystem, the forces driving adoption are very strong at 5 out of 5.
Diffusion (score: 4)
Demand: While chiplets provide benefits, the approach also faces adoption barriers. It increases design complexity with teams optimizing chiplets and interconnects. Moving to chiplets requires semiconductor firms to adopt new architectures, design methodologies and tools. This slows adoption until skills and tools mature. Testing and qualifying assembled chiplets increases verification complexity. Monolithic chip designs currently meet most performance needs, limiting demand. High bandwidth density interconnects between chiplets remain challenging. Favoring simplicity and familiarity, risk-averse customers may prefer traditional monolithic chip designs.
Supply: On the supply side, the chiplet ecosystem is still evolving. Lack of standards increases costs and risks. Although the standardisation process is developing through the CHIPS Alliance open consortium launched in 2019 aims to develop common interfaces for chiplets; DARPA's Common Heterogeneous Integration and IP Reuse Strategies (CHIPS) program has defined standardized chiplet design elements like die-to-die interfaces; and the IEEE has chiplet related standards projects in development including P1838 on multi-die interconnect and P2452 for embedded die in substrate integration. Despite progress on standards, the industry still lacks software that runs across heterogeneous chips making design easy. More R&D is needed to improve interconnect density and heterogeneous integration. Foundries are structured and optimized for monolithic chip fabrication. Transitioning foundry models and operations to support chiplets will take restructuring and investment. Despite the not insignificant barriers, they will be addresses as the market grows and matures.
Impact (score: 3)
The impact of widespread chiplet adoption could range from incremental to highly disruptive. In a low impact scenario, chiplets simply provide incremental flexibility versus monolithic chips for high-end applications. This would represent only marginal industry change.
However, in a high impact scenario, chiplets could transform chip design architecture across multiple segments from HPC to mobile and IoT. Modular integration and mixing libraries could enable new classes of hybrid chips. We’ve talked about the likelihood of hybrid optoelectronic and analog-electronic devices in previous editions. Manufacturing chipsets from pre-designed chiplets could reshape foundry models and lower barriers for new entrants. Interconnect standards could enable an open chiplet marketplace. Such a transition from monolithic to modular integration would profoundly reshape the semiconductor industry.
On balance, chiplets will likely have a moderately high impact, driving more customization and heterogeneity in chip designs across HPC, AI, networking and other segments. They are an enabling technology for optical and analog co-processors. Chiplets are an evolution not revolution. It is somewhat similar to RISCV in that is will come to dominate the market but is unlikely to change the value chain materially. A score of 3 out of 5 balances the incremental and disruptive possibilities.
Timing (2020-2025)
In 2021, the global chiplet market was valued at around $3 billion but is forecast to grow rapidly to $20 billion by 2027 at a CAGR of 40%. Chiplets are already being incorporated into commercial system-on-a-chip (SoC) products. Major chipmakers including AMD, Intel, Nvidia, Marvell, and SiFive have announced or shipped SoCs using chiplets for enterprise CPUs, AI accelerators, networking chips, and other applications requiring high performance and customization. Products such as AMD’s Epyc Rome server processors, Intel’s Lakefield consumer CPUs, and Nvidia’s HGX AI accelerators demonstrate early adoption of modular chiplet architectures in order to integrate diverse functionality and process nodes onto a single advanced package.
Under/Overrated
Correctly Rated. As an incremental efficiency and customization improvement, chiplets are correctly rated. Although as an enabling technology for hybrid/mixed signal chips, potentially underrated.
Open Questions
Will design complexity and the lack of standards impede widespread adoption outside of high-end applications? How do we create an abstraction of a chiplet that’s stable across design in the same way as a SPICE model or a standard cell library?
Can interconnect density continue improving to approach intrasystem bandwidths?
Since TSMC announced a cloud alliance in 2018 and moved their 5nm simulation to the cloud, the EDA industry is slowly moving to the cloud. Is cloud migration a catalyst for chiplet growth as hardware is no longer a bottleneck? (5nm simulation, or any SoC in general, it takes 5X to 10X the hardware resources to be able to do that versus a 16/14nm chip)
2030 Prediction
Captures 8% of semiconductor architecture market share
Opportunities:
Photonic Interconnects
There is an opportunity for startups to develop high-speed photonic interconnect solutions between chiplets. Using light rather than electrical signals, photonic interconnects can provide much higher bandwidths exceeding terabits/second to alleviate IO bottlenecks. This allows optimizing chiplets for different functions and seamlessly connecting them. Startups can focus on cost-effective silicon photonics devices, packaging, laser light sources, and tools to integrate photonics within chiplet architectures. Intel, AMD, Xilinx, TSMC and IMEC are already investing in chiplet photonics, but a startup with deep optics experience could compete. (See Ayar Labs, Black Semiconductor, Eliyan, Scintil Photonics)
ML-based EDA Tools
Another opportunity is machine learning powered electronic design automation (EDA) tools tailored for chiplet-based designs. Complexity is a key barrier, with thousands of chiplets needing optimized placement/routing and co-simulation. Startups can apply graph neural networks, reinforcement learning, and other ML techniques to automate chiplet floor-planning, verification, interconnect optimization and more. This will make chiplet design accessible to a wider group of companies. Startups can target specific EDA pain points like physical implementation or analog/mixed-signal analysis for chiplet architectures.
Mixed-Signal/Hybrid Chiplets
Startups can create design libraries and standard interfaces around hybrid chiplets that integrate multiple technologies like analog, digital, RF, sensors, MEMS, optics, and even quantum. Assembling these diverse chiplets can enable new applications and on-chip heterogeneity. But they present integration challenges between different process nodes. Startups focused on mixed-signal chiplets can advance precision packaging, novel substrates, and modeling tools needed to bridge different components into advanced System-in-Package solutions. This will expand the use cases for chipletization.
[Redacted]
If you are still here, you must have got some value? Return the favour and then enjoy the GIF.
Obviously, technically I’m a venture capitalist. So get in touch if you are a founder? Or maybe you invest money? Maybe you get in touch? I have people who can do VC etc. Email me lawrence@lunar.vc and I will put in in touch with a professional.
Sources
https://www.eetimes.com/chiplets-a-short-history/
https://semiengineering.com/knowledge_centers/packaging/advanced-packaging/chiplets/
https://semiengineering.com/chiplets-gaining-steam/
https://semiengineering.com/eda-gaps-at-the-leading-edge/
https://semiengineering.com/bridging-the-chiplet-gap/
https://www.darpa.mil/program/common-heterogeneous-integration-and-ip-reuse-strategies
https://eps.ieee.org/technology/heterogeneous-integration-roadmap.html