Citi | US Semiconductors: AI Infrastructure Summit Day 2: The Network Is the Computer
Summaries are public. PDF access requires an active membership; all members have the same access. Sign in
Citi semiconductor analysts Atif Malik and James Bowlin attended Day 2 of the AI Infrastructure Summit in Santa Clara, with networking the focus. Their central conclusion: as AI workloads grow more inference-intensive and context-rich, scaling infrastructure requires innovation across the memory hierarchy, HBM, system architecture and network fabrics, not just more compute. Broadcom argued Ethernet is becoming the interconnect of choice for large-scale AI, with Tomahawk roadmap bandwidth doubling steadily—Tomahawk 6 reaches 102.4 Tbps to support larger AI clusters—and claims a 5-year TCO advantage above 30% versus alternatives. Marvell highlighted fast-growing KV cache as the key inference challenge: larger context windows can exceed HBM capacity, forcing reloads or recomputation from slower memory tiers and raising latency, memory traffic and power. Samsung expects the evolution from generative to agentic and physical AI to lift training infrastructure needs roughly 3x and inference roughly 100x; HBM performance roughly doubles per generation (HBM4E at 16 TB/s per GPU; HBM5 double HBM4E; zHBM 4-8x HBM5), and Z-NAND samples in 2028. AWS traced its custom-silicon path from Annapurna Labs (2015) through Graviton, Inferentia and Trainium 4, emphasizing memory bandwidth per dollar and Rainier clusters. No ratings or targets were given.
Key exhibits
Exhibit 1
