GenAI Infrastructure Part 1.

What are we talking about? A Google search for “what is GenAi” got ~153,000,000 results. Wikipedia has a pretty good article which is part of a larger series on AI in general.

I think we are in the world of Blind men and an elephant.

Recently I’ve been looking into “AI Networking” which is the coolest, latest thing for the networking industry. I’m a bit of a skeptic about GenAI because I don’t understand where the revenue is coming from to pay for all the infrastructure (e.g., DC physical infrastructure, network infrastructure, GPUs, TPUs) that is being built today, e.g.:

• Dell'Oro on Data Center Capex

• Three Mile Island nuclear plant will reopen to power Microsoft data centers

• Is the rise of genAI about to create an energy crisis?

• AI’s $600B Question

I’ll come back to that topic sometime later.

Good Stuff to Read

I’ve been reading up and I thought I’d share some of the sources I’ve found useful.

SemiAnalysis

The folks at SemiAnalysis have some really good articles available on SubStack. They have a free level and a $500 annual subscription. Here are some of the articles I really liked:

• Datacenter Anatomy Part 1: Electrical Systems

• AI Neocloud Playbook and Anatomy

• Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructure

•  100,000 H100 Clusters: Power, Network Topology, Ethernet vs InfiniBand, Reliability, Failures, Checkpointing

• AI Datacenter Energy Dilemma - Race for AI Datacenter Space

 2024 OCP Global Summit

I’m starting to pay attention to OCP because the system constrains are going to drive what’s possible in the networking space. I didn’t get to attend the 2024 Global Summit in San Jose, but I’ve been looking at the slides and some of the videos. The Keynotes and the Networking tracks are probably the first areas to look at, but some of the stuff in Rack & Power and Cooling Environments is pretty wild.

Specific sessions I’ve been looking at include (links point to videos):

• Keynotes:

  • Powering the AI Future: Meta Vision for Open Systems for AI

  • Microsoft: From Bottlenecks to Breakthroughs: Innovations in Data Center Infrastructure & Security

  • Google: Sustainable Silicon to Intelligent Clouds: Building the Compute Infrastructure of the Future

• Networking:

  • Meta 51.2T Ethernet Switch

• Rack & Power:

  • +/-400VDC Rack Power System for Ml/AI Application

  • Meta ORV3N (Open Rack V3 Networking) Solution              

FYI - You can find playlists for all the tracks at OCP playlists.

WebScaler blogs etc

The WebScalers have a lot of good stuff online:

• Google

  • About Google Data Centers

  • TPU transformation: A look back at 10 years of our AI-specialized chips

  • Why Google keeps building custom silicon: The story behind Axion

  • Speed, scale and reliability: 25 years of Google data-center networking evolution

  • Falcon, a reliable low-latency hardware transport

• Meta

  • Innovation - Meta Data Centers

  • Building Meta’s GenAI Infrastructure

  • Meta’s open AI hardware vision

  • Maintaining large-scale AI capacity at Meta

  • OCP Summit 2024: The open future of networking hardware for AI

• Microsoft

  • Mt Diablo - Disaggregated Power Fueling the Next Wave of AI Platforms

  • Liquid Cooling in Air Cooled Data Centers on Microsoft Azure       

  • Inside Maia 100: Revolutionizing AI Workloads with Microsoft's Custom AI Accelerator

  • The anatomy of a datacenter

To be continued.