SGLang and Miles Deliver Day-One Support for Moonshot AI 2.8T-Parameter Kimi K3 Model

Key Highlights

SGLang and Miles announced full support for Moonshot AI’s Kimi K3 on the very day of its release, taking on the two critical pipelines of inference deployment and reinforcement-learning training respectively. For a freshly released open-weight large model, this kind of “available on day one” speed is quite rare, and it sends a clear message: domestic inference and training frameworks have caught up with the iteration pace of frontier models rather than lagging weeks behind.

The significance goes beyond mere convenience. When a model ships with working tooling from hour one, the community can start benchmarking, fine-tuning, and productizing immediately instead of stalling while waiting for compatibility fixes. That compression of the adoption cycle is exactly what separates a model that merely launches from one that actually lands in production, and it is a strong sign of ecosystem maturity. It also means third-party tutorials, adapters, and enterprise pilots can be prepared in advance, so the moment the weights drop, the whole chain is ready.

What Happened

Kimi K3 is an open-weight 2.8T-parameter model from Moonshot AI. On its release day, SGLang delivered efficient inference support, while Miles simultaneously filled in the RL training side. In other words, developers could run inference on SGLang and do reinforcement-learning fine-tuning on Miles at the very first moment the model appeared, without waiting for community patches after the fact.

Behind this day-one support lies close coordination and early adaptation between the two teams before the launch, not a last-minute scramble. Such readiness typically requires the framework maintainers to have had early access to the model specification, to have prototyped the kernels, and to have validated the numerics against the reference implementation. That level of pre-release alignment is a strong indicator of how tightly the Chinese open-source AI ecosystem now operates, and it lowers the friction for anyone who wants to build on top of K3. For Moonshot AI, shipping with SGLang and Miles already wired up is also a distribution strategy: it lowers the activation energy for adoption and makes K3 the path of least resistance for new projects.

Technical Details

K3’s architecture is genuinely interesting: it adopts a hybrid structure interleaving 69 layers of KDA linear attention with 24 layers of MLA. It is neither pure linear attention nor the dense attention of a traditional Transformer, but rather an interleaving of the two mechanisms within a single network, which keeps both long-range efficiency and local expressiveness in play at the same time. Keeping both properties simultaneously is the architectural bet here; many hybrids sacrifice one for the other, yet K3 tries to avoid that trade.

This design yields direct speed benefits. On SGLang, single-card batch-1 decode speed reaches about 113 tok/s. Combined with speculative decoding, throughput can be pushed further to about 423 tok/s. For applications that need long context and low latency, these numbers are quite competitive, because they translate into lower per-request cost and snappier interactive experiences. The hybrid design also means the model does not force users to choose between context length and inference efficiency; it tries to keep both, which is why it suits real production workloads rather than just benchmark runs, and the fact that a 2.8T-parameter model stays single-card viable is the practical headline that expands who can afford to serve it.

vs Competitors

Placing K3 among comparable open-weight models, its hybrid architecture clearly departs from the conventional route of “stacking parameters, going dense.” Linear attention brings down the compute for long sequences, while MLA preserves the expressive power of key information; their interleaving lets the model stay usable on a single card even at a staggering parameter scale, which is the property that makes deployment realistic for smaller teams.

Compared with dense large models that require expensive clusters to run, the K3-plus-SGLang combination is much closer to real engineering deployment. Other open-weight releases often arrive with inference support that needs days or weeks of community effort to stabilize; K3’s day-one readiness sets a higher bar for what users should expect from a launch, and it rewards teams that invest in early integration. It also reframes the launch as a platform moment rather than a single-model event, since the surrounding tooling is what turns weights into workflows.

Industry Impact

Put simply, day-one support itself is a signal: domestic inference and training frameworks are moving from “following behind” toward “running alongside” or even “keeping pace” with frontier models. For enterprises and researchers, K3’s open weights plus ready-to-use frameworks mean frontier capability can be brought into their own businesses at controllable cost—for long-context inference and RL post-training—rather than staying at the demo stage.

As more frontier models ship with same-day framework support, the bottleneck shifts from “can I run it” to “what should I build with it,” which is arguably the healthier place for an ecosystem to be. The K3 case shows that the supporting tooling, not just the weights, is becoming a competitive dimension in its own right, and the real leverage in the open-weight race may sit as much in the ecosystem around the model as in the model itself.