Groq

Overview

Groq is an inference-chip company used across the sources as the leading example of SRAM-heavy, compiler-centered, deterministic low-latency specialization. Its architecture moves much scheduling work before runtime and keeps data close to compute, trading generality and capacity for predictable execution and fast token delivery.

Current Profile

The current profile is technically differentiated but workload-bounded. Earlier sources position Groq as a low-latency complement rather than a universal Nvidia or TPU replacement. E251 explains the mechanism more precisely: compiler-time schedules and on-chip SRAM reduce runtime uncertainty and memory movement, while dynamic MoE routing can force over-communication or idle hardware. EP36 repeats the acquisition and rack-scale LPU narrative, but its “Grok” spelling and “Grok 3” product label are treated as transcription uncertainty rather than new canonical identity evidence.

Key Characteristics

• Uses an SRAM-heavy design to bring model data close to compute and reduce memory-access variability.
• Centers hardware design on deterministic, ahead-of-time compiler scheduling.
• Targets latency-sensitive inference such as agents, real-time voice, and serial model-call workflows.
• Faces efficiency pressure when runtime-dependent workloads such as MoE do not fit static schedules cleanly.
• Competes at system and software level, not through isolated chip specifications alone.

Evidence

• Low-latency niche: E230 uses Groq to show where communication-efficient inference can matter despite Nvidia’s full-stack moat.
• Market segmentation: E228 contrasts Groq’s single-user latency orientation with TPU-favorable, high-throughput batching.
• Architecture and qualification: E251 connects deterministic SRAM execution to decode bandwidth and explains the mismatch between static schedules and dynamic expert routing.
• GTC recap: EP 36: NVIDIA GTC 2026: Everything That Matters - Recapped describes Nvidia presenting a rack-scale LPU system after a reported acquisition; the source’s “Grok” spelling is normalized to Groq.

Qualifications

The sources do not provide independently comparable cost, latency, utilization, or energy benchmarks. Groq’s advantage depends on model shape, batching, interconnect, compiler quality, and acceptable idle capacity. The reported Nvidia acquisition, its price, strategic rationale, “Groq 3” generation name, and 256-LPU rack configuration remain source-scoped rather than independently verified here.

What Changed

• Migrated the page to the synthesis-v1 entity schema.
• Added the deterministic compiler and SRAM mechanism behind the low-latency profile.
• Added MoE runtime routing as a concrete qualification and the reported acquisition as source-scoped context.
• Normalized EP36’s “Grok” wording to Groq and retained its product and rack claims as uncertain source detail.

Relationships

• Low-Latency Inference Chip - chip category for Groq’s latency-oriented specialization.
• Inference Decode Bandwidth - data-movement bottleneck its SRAM and scheduling choices target.
• Mixture of Experts - dynamic workload pattern that strains static schedules.
• Nvidia - incumbent and reported acquirer in the newest source.
• TPU - high-throughput specialized accelerator contrasted with Groq’s latency niche.
• AI Chip Specialization - broader tradeoff between workload fit and flexibility.