CoreWeave Achieves New Record-Breaking AI Inferencing Benchmark with NVIDIA GB200 Grace Blackwell Superchips

CoreWeave is the first cloud service provider to submit MLPerf Inference v5.0 results for NVIDIA GB200 Superchips

CoreWeave, the AI Hyperscaler™, today announced its MLPerf v5.0 results, setting a new industry benchmark in AI inference with NVIDIA GB200 Grace Blackwell Superchips. Using a CoreWeave instance with NVIDIA GB200, featuring two NVIDIA Grace CPUs and four NVIDIA Blackwell GPUs, CoreWeave delivered 800 tokens per second (TPS) on the Llama 3.1 405B model¹—one of the largest open-source models.

Also Read: How AI can help Businesses Run Service Centres and Contact Centres at Lower Costs?

“CoreWeave is committed to delivering cutting-edge infrastructure optimized for large-model inference through our purpose-built cloud platform,” said Peter Salanki, Chief Technology Officer at CoreWeave. “These benchmark MLPerf results reinforce CoreWeave’s position as a preferred cloud provider for leading AI labs and enterprises.”

CoreWeave also submitted new results for NVIDIA H200 GPU instances. It achieved 33,000 TPS on the Llama 2 70B model, representing a 40 percent improvement in throughput over NVIDIA H100 instances.²

These results further demonstrate CoreWeave as an industry-leading cloud infrastructure services provider. This year, the company became the first to offer general availability of NVIDIA GB200 NVL72-based instances. Last year, the company was among the first to offer NVIDIA H100 and H200 GPUs, and it was one of the first to demo NVIDIA GB200 NVL72.

MLPerf Inference is an industry-standard suite for measuring machine learning performance across realistic deployment scenarios. How quickly systems can process inputs and produce results using a trained model has a direct impact on user experience.

Latest Read: Taking Generative AI from Proof of Concept to Production

CoreWeave, the AI Hyperscaler™, delivers a cloud platform of cutting-edge software powering the next wave of AI. The company’s technology provides enterprises and leading AI labs with cloud solutions for accelerated computing. Since 2017, CoreWeave has operated a growing footprint of data centers across the US and Europe. CoreWeave was ranked as one of the TIME100 most influential companies and featured on Forbes Cloud 100 ranking in 2024.

[To share your insights with us as part of editorial or sponsored content, please write to psen@itechseries.com]