Modal, Fireworks, and Baseten gain cost advantage with Nvidia and AMD chips for Kimi K3

Three American AI infrastructure companies have found a lucrative edge in an unlikely place: serving a Chinese-built model better and cheaper than its creators can. Modal, Fireworks AI, and Baseten are now offering hosted inference for Moonshot AI’s Kimi K3 at roughly one-tenth the cost of direct access, powered by Nvidia’s and AMD’s latest accelerator hardware that remains largely unavailable to Chinese firms under US export controls.

What makes Kimi K3 worth the effort

Kimi K3 is not a small model. Released by Moonshot AI on July 27, 2026, it packs 2.8 trillion parameters into a Mixture-of-Experts architecture, making it one of the largest open-weights models ever published. The “open weights” distinction matters: anyone can download and run the model, which is exactly what Modal, Fireworks, and Baseten have done.

The MoE design means the full 2.8 trillion parameters don’t activate on every query. Instead, K3 routes each token through 16 of its 896 available experts, activating roughly 104 billion parameters per forward pass. K3 also supports a context window of 1 million tokens, which means it can ingest and reason over book-length documents in a single session.

But running a model this size is not trivial. The weight files exceed 1.4 terabytes even when compressed using MXFP4 quantization. Baseten has noted that deployments require multi-GPU configurations, with setups like eight Nvidia GB300 accelerators working in concert just to hold the model in memory.

The hardware advantage

US inference providers have access to Nvidia’s GB300 NVL72 systems and AMD’s MI350X and MI355X accelerators. Chinese companies, including Moonshot AI itself, face export restrictions that limit their access to these exact processors.

Modal reported that its deployments of Kimi K3 achieve 460 tokens per second using a custom speculative decoder called DFlash, purpose-built for K3’s architecture. The pricing reflects these efficiencies. Shared endpoints across the three providers land at approximately $3 per million input tokens, $0.30 per million cached tokens, and $15 per million output tokens.

Compliance as a selling point

Beyond raw performance, Modal, Fireworks, and Baseten are marketing something arguably more valuable to their target customers: data residency and compliance guarantees. All three emphasize zero-data retention policies, meaning customer prompts and completions are not stored or used for training. They also offer dedicated capacity and US-hosted endpoints, features that matter enormously to enterprises navigating data sovereignty regulations in sectors like healthcare, finance, and government contracting.

The three providers also support OpenAI-compatible APIs, which lowers the switching cost for enterprises already integrated with OpenAI’s tooling. A developer can point their existing code at a Kimi K3 endpoint with minimal changes, testing whether the 2.8 trillion-parameter model outperforms their current setup without a major engineering investment.

Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy.