Nvidia said on August 24 that its Groq 3 LPX rack is in full production, after its US$20 billion deal for Groq assets.

Nvidia said Nebius, a Netherlands-based cloud provider, will deploy the system later this year alongside Nvidia’s Vera central processing units and Rubin graphics processing units for low-latency AI inference.

Nvidia said each rack contains 256 Groq 3 chips manufactured by Samsung.

Citing an Artificial Analysis benchmark, Nvidia said the system delivers 3,400 tokens per second.

It added that OpenAI’s Ultrafast mode is listed at 750 tokens per second and runs on Cerebras.

Nvidia said demand is growing for chips that handle the decode stage of inference, which helps generate model outputs quickly.

The company said the Groq system would complement graphics processing units rather than replace them.

🔗 Source: CNBC

Recent Nvidia developments