Groq vs Tenstorrent

Architectural Philosophy: Determinism vs Flexibility

Groq's LPU feeds tokens through a single wide pipeline of functional units executing in lock-step—no kernel switching, no cache misses, every clock cycle doing useful work. This deterministic approach eliminates the scheduling overhead that plagues GPU and TPU architectures, delivering predictable latency that matters enormously for agentic AI applications where multiple LLM calls chain within a single interaction. Tenstorrent's Tensix cores take the opposite approach: a mesh-based architecture with conditional execution that can skip unnecessary computation dynamically. This flexibility lets the same hardware handle training and inference workloads, and the RISC-V instruction set means the architecture can be extended and customized by licensees without proprietary lock-in.

The NVIDIA Factor

NVIDIA's $20 billion acquisition of Groq in December 2025 fundamentally changed the competitive landscape. The Groq 3 LPU, unveiled at GTC 2026, claims 35x higher throughput per megawatt than NVIDIA's own Blackwell NVL72 for trillion-parameter models—NVIDIA essentially acquired the technology that would obsolete its own inference approach. Jonathan Ross and Groq's senior engineering team now lead NVIDIA's Real-Time Inference division. For Tenstorrent, this consolidation is a double-edged sword: it removes an independent competitor but also validates the thesis that specialized inference hardware is critical. Jim Keller's strategy of building the anti-NVIDIA—open architecture, no HBM dependency, IP licensing—becomes more differentiated as NVIDIA absorbs Groq's technology into its walled garden.

Memory and Cost Architecture

Both companies reject HBM (High Bandwidth Memory), but for different reasons. Groq uses only on-die SRAM to achieve extreme bandwidth (150 TB/s on Groq 3) at the cost of limited capacity—the architecture works brilliantly for inference but cannot scale to training-sized models without multi-chip configurations. Tenstorrent uses commodity GDDR6 DRAM paired with a distributed software memory architecture, sacrificing peak bandwidth for dramatically lower bill-of-materials cost. Manufacturing on Samsung's SF4X process rather than TSMC's cutting-edge nodes further reduces Tenstorrent's cost basis. If Groq 3 represents the performance ceiling of AI inference silicon, Tenstorrent's Blackhole represents the cost floor—and in an inference economy where margins matter, both positions are viable.

Software Ecosystem Maturity

Post-acquisition, Groq benefits from NVIDIA's CUDA ecosystem—the largest and most mature GPU software stack in the world. Developers already using TensorRT for inference can potentially target Groq 3 LPUs with minimal code changes. Tenstorrent's TT-Metalium is open-source and growing, but remains early-stage compared to CUDA's decade-plus head start. Jim Keller has acknowledged this gap, noting that Tenstorrent needs 18–36 months of software stack maturation to achieve meaningful market traction. The open-source approach could ultimately be an advantage—much as Linux eventually displaced proprietary Unix—but the near-term developer experience favors NVIDIA/Groq.

Target Markets and Deployment Models

Groq (now under NVIDIA) targets hyperscale cloud providers and enterprises demanding the absolute fastest inference for real-time agentic web applications—the 1,500 tokens-per-second target enables multi-agent systems communicating in real time. Tenstorrent targets a broader market including edge deployment, sovereign AI initiatives, and organizations that want to customize their AI silicon without NVIDIA dependency. Tenstorrent's developer workstations starting at $12,000 and IP licensing model enable a long tail of hardware customization that Groq/NVIDIA's vertically integrated approach cannot serve. The Taiwan office expansion signals Tenstorrent's ambition to embed in the global semiconductor supply chain.

The Open Architecture Bet

Tenstorrent's use of RISC-V aligns with a broader industry shift toward open-source foundations in AI infrastructure. Just as open-source models from Meta and Mistral challenged proprietary LLMs, open-source hardware architectures challenge NVIDIA's proprietary GPU stack. Tenstorrent's licensing model—selling IP and chiplets that others can integrate—mirrors ARM's approach to mobile computing. If RISC-V AI accelerators achieve even a fraction of ARM's mobile success, Tenstorrent's early positioning becomes enormously valuable. Groq's absorption into NVIDIA forecloses this path entirely: its technology is now proprietary NVIDIA IP, accessible only through NVIDIA's product stack and pricing.