Gimlet & Cerebras to deliver AI inference at speed

Gimlet and Cerebras have partnered to deliver AI inference through Gimlet Cloud, with a stated goal of speeds of up to 3,000 tokens per second.

The agreement adds Cerebras as a supported accelerator within Gimlet's cloud service, which Gimlet describes as a multi-silicon inference platform. The company plans to deploy 100 MW of Cerebras-powered inference capacity, and the integrated system is already operating in private deployments.

The setup combines the Cerebras Wafer Scale Engine with GPUs in a disaggregated inference architecture. Gimlet's software directs different phases of model execution to the type of silicon best suited to each task, from datacentre infrastructure to developer-facing application programming interfaces.

Target use cases include agentic systems and real-time applications such as voice, video, assistants, and other interactive services where response speed shapes the user experience.

Private deployments

The partnership builds on joint work with customers that has been under way for several quarters. According to the companies, that work has already produced an integrated deployment serving tokens in private environments.

That existing deployment gives the announcement more weight than a standard product tie-up, showing the software and hardware combination has already moved beyond testing. The next step is broader integration across software, infrastructure design, optimisation, validation, production operations, and developer tools.

Gimlet, a San Francisco-based company backed by Andreessen Horowitz and Menlo Ventures, focuses on software for AI inference across different chip architectures. Cerebras, which is listed on Nasdaq, is known for wafer-scale processors designed for AI workloads.

Speed claims

The companies presented the performance target as a significant increase over conventional fast inference services, which they said typically run in the hundreds of tokens per second rather than the thousands. In practice, token throughput is a common benchmark for how quickly a large language model can generate or process text.

For AI service providers, faster output can affect how conversational or responsive an application feels, particularly in systems designed for live exchanges rather than batch processing. Pairing Cerebras hardware with GPU-based throughput also reflects a wider trend in AI infrastructure towards combining specialised processors rather than relying on a single chip family.

"Inference speed matters. It determines how productive AI can be. Fast inference creates magical user experiences and opens new markets. By combining Gimlet's multi-silicon software with the Cerebras Wafer Scale Engine, we can run each phase of inference on the hardware best suited to it and plan to deliver up to 3,000 tokens per second at production scale," said Zain Asgar, Co-Founder and Chief Executive Officer, Gimlet Labs.

The collaboration will also make Cerebras' latest CS-4 systems available through Gimlet, with Gimlet serving as a launch partner.

"Combining the fastest tokens from Cerebras with the highest throughput GPUs delivers the best datacenter economics for everyone," said Sean Lie, Co-Founder and Chief Technology Officer, Cerebras.

He added: "Everyone wants more high-value tokens. Cerebras delivers the fastest AI inference in the world, and GPUs deliver high throughput. By making Cerebras a native part of its inference cloud, Gimlet will bring our industry-leading speed and intelligent AI to more developers at production scale. We're excited to build with Gimlet as a launch partner for CS-4, giving customers a direct path to our latest technology."