Groq Teardown — Jonathan Ross's $2.8B LPU Bet vs NVIDIA on Inference Speed

Verdict First

Groq is one of the few AI infrastructure companies I think genuinely has a shot at carving real estate out of NVIDIA's wall — but not for the reason most people on Twitter are screaming about. The "500 tokens per second" demos are a marketing wrapper around something more boring and more durable: a deterministic compute architecture that turns inference latency into a contract instead of a guess. If you've ever shipped a voice agent and watched p99 latency wreck your retention curve, you already understand why this matters more than the raw speed number.

You, the indie hacker reading this, cannot replicate Groq. The capital wall is between $300M and $1B depending on how you count the tapeouts. The team wall is harder — you need ex-Google TPU silicon people, and there are maybe 200 of them globally.

What you should walk away with: a clear understanding of which application categories are now economically viable because Groq exists, and why an 18-month window is opening for builders who pick the right speed-sensitive vertical. Voice agents under 300ms round trip. Real-time translation for live calls. NPC dialog in games where every token of delay breaks immersion. These are the wedges. Groq is the picks-and-shovels. You're the prospector.

Quick Facts

Field	Value
Product	Groq (GroqCloud API + LPU silicon)
Founded	2016
Founder	Jonathan Ross (ex-Google, original TPU architect)
HQ	Mountain View, California
Stage	Series D, $640M raised August 2024
Valuation	$2.8B post-money
Lead Investor (D)	BlackRock Private Equity Partners
Reported MRR	$4M+ (Aug 2024 baseline)
Business Model	API-met Sign in to read this report You have read your 1 free report. Sign in with Google to unlock 2 more. Sign in with Google