Cerebras Systems, a pioneer in high-performance AI compute, has introduced a groundbreaking solution that is set to revolutionize AI inference. On August 27, 2024, the company announced the launch of Cerebras Inference, the fastest AI inference service in the world. With performance metrics that dwarf those of traditional GPU-based systems, Cerebras Inference delivers 20 times the speed at a fraction of the cost, setting a new benchmark in AI computing.
Unprecedented Speed and Cost Efficiency
Cerebras Inference is designed to deliver exceptional performance across various AI models, particularly in the rapidly evolving segment of large language models (LLMs). For instance, it processes 1,800 tokens per second for the Llama 3.1 8B model and 450 tokens per second for the Llama 3.1 70B model. This performance is not only 20 times faster than that of NVIDIA GPU-based solutions but also comes at a significantly lower cost. Cerebras offers this service starting at just 10 cents per million tokens for the Llama 3.1 8B model and 60 cents per million tokens for the Llama 3.1 70B model, representing a 100x improvement in price-performance compared to existing GPU-based offerings.
Maintaining Accuracy While Pushing the Boundaries of Speed
One of the most impressive aspects of Cerebras Inference is its ability to maintain state-of-the-art accuracy while delivering unmatched speed. Unlike other approaches that sacrifice precision for speed, Cerebras’ solution stays within the 16-bit domain for the entirety of the inference run. This ensures that the performance gains do not come at the expense of the quality of AI model outputs, a crucial factor for developers focused on precision.
Micah Hill-Smith, Co-Founder and CEO of Artificial Analysis, highlighted the significance of this achievement: “Cerebras is delivering speeds an order of magnitude faster than GPU-based solutions for Meta’s Llama 3.1 8B and 70B AI models. We are measuring speeds above 1,800 output tokens per second on Llama 3.1 8B, and above 446 output tokens per second on Llama 3.1 70B – a new record in these benchmarks.”
The Growing Importance of AI Inference
AI inference is the fastest-growing segment of AI compute, accounting for approximately 40% of the total AI hardware market. The advent of high-speed AI inference, such as that offered by Cerebras, is akin to the introduction of broadband internet—unlocking new opportunities and heralding a new era for AI applications. With Cerebras Inference, developers can now build next-generation AI applications that require complex, real-time performance, such as AI agents and intelligent systems.
Andrew Ng, Founder of DeepLearning.AI, underscored the importance of speed in AI development: “DeepLearning.AI has multiple agentic workflows that require prompting an LLM repeatedly to get a result. Cerebras has built an impressively fast inference capability which will be very helpful to such workloads.”
Broad Industry Support and Strategic Partnerships
Cerebras has garnered strong support from industry leaders and has formed strategic partnerships to accelerate the development of AI applications. Kim Branson, SVP of AI/ML at GlaxoSmithKline, an early Cerebras customer, emphasized the transformative potential of this technology: “Speed and scale change everything.”
Other companies, such as LiveKit, Perplexity, and Meter, have also expressed enthusiasm for the impact that Cerebras Inference will have on their operations. These companies are leveraging the power of Cerebras’ compute capabilities to create more responsive, human-like AI experiences, improve user interaction in search engines, and enhance network management systems.
Cerebras Inference: Tiers and Accessibility
Cerebras Inference is available across three competitively priced tiers: Free, Developer, and Enterprise. The Free Tier provides free API access with generous usage limits, making it accessible to a broad range of users. The Developer Tier offers a flexible, serverless deployment option, with Llama 3.1 models priced at 10 cents and 60 cents per million tokens. The Enterprise Tier caters to organizations with sustained workloads, offering fine-tuned models, custom service level agreements, and dedicated support, with pricing available upon request.
Powering Cerebras Inference: The Wafer Scale Engine 3 (WSE-3)
At the heart of Cerebras Inference is the Cerebras CS-3 system, powered by the industry-leading Wafer Scale Engine 3 (WSE-3). This AI processor is unmatched in its size and speed, offering 7,000 times more memory bandwidth than NVIDIA’s H100. The WSE-3’s massive scale enables it to handle many concurrent users, ensuring blistering speeds without compromising on performance. This architecture allows Cerebras to sidestep the trade-offs that typically plague GPU-based systems, providing best-in-class performance for AI workloads.
Seamless Integration and Developer-Friendly API
Cerebras Inference is designed with developers in mind. It features an API that is fully compatible with the OpenAI Chat Completions API, allowing for easy migration with minimal code changes. This developer-friendly approach ensures that integrating Cerebras Inference into existing workflows is as seamless as possible, enabling rapid deployment of high-performance AI applications.
Cerebras Systems: Driving Innovation Across Industries
Cerebras Systems is not just a leader in AI computing but also a key player across various industries, including healthcare, energy, government, scientific computing, and financial services. The company’s solutions have been instrumental in driving breakthroughs at institutions such as the National Laboratories, Aleph Alpha, The Mayo Clinic, and GlaxoSmithKline.
By providing unmatched speed, scalability, and accuracy, Cerebras is enabling organizations across these sectors to tackle some of the most challenging problems in AI and beyond. Whether it’s accelerating drug discovery in healthcare or enhancing computational capabilities in scientific research, Cerebras is at the forefront of driving innovation.
Conclusion: A New Era for AI Inference
Cerebras Systems is setting a new standard for AI inference with the launch of Cerebras Inference. By offering 20 times the speed of traditional GPU-based systems at a fraction of the cost, Cerebras is not only making AI more accessible but also paving the way for the next generation of AI applications. With its cutting-edge technology, strategic partnerships, and commitment to innovation, Cerebras is poised to lead the AI industry into a new era of unprecedented performance and scalability.
For more information on Cerebras Systems and to try Cerebras Inference, visit www.cerebras.ai.
