0 支持 NVIDIA Nemotron 3.5 Lightning
Core Highlights
The SGLang project team announced that it now provides day-zero support for NVIDIA's newly released Nemotron 3.5 Lightning model. In practical terms, this means developers can deploy and invoke the model through the SGLang inference framework on the very same day the model becomes available, with no need to wait for separate third-party compatibility work to catch up. Nemotron 3.5 Lightning is an open-source mixture-of-experts (MoE) model whose total parameter count reaches 30B, yet only 3B of those parameters are activated during each individual inference step. This design preserves the capability that one would normally expect from a much larger model, while at the same time significantly reducing the cost of running inference at scale. By keeping the active parameter footprint deliberately small, the model stays affordable to serve even when context lengths grow very large, which is exactly the situation where dense models tend to become expensive and difficult to operate for smaller teams that lack rack-scale accelerators and dedicated infrastructure.
What Happened
According to the official announcement, Nemotron 3.5 Lightning supports an exceptionally long context window of up to 1M tokens. That capacity makes it well suited to demanding tasks such as processing entire books from start to finish, navigating very large codebases, or summarizing long documents in a single pass without fragmentation. Users can download weight variants in two precisions, BF16 and NVFP4, directly from the Hugging Face repository, which lets them pick the format that best matches the hardware they already own. On top of that, the model supports three distinct speculative decoding techniques, named MTP, DFlash, and DSpark. These techniques raise generation throughput by predicting several tokens at once instead of generating them one at a time, which helps a great deal when many requests arrive together and the serving stack is under real production load from actual users.
Technical Details
At the architectural level, Nemotron 3.5 Lightning relies on a mixture-of-experts design, and of its 30B total parameters only 3B are activated per step, which is precisely what makes single-GPU deployment realistic for many teams. The model is published in both BF16 and NVFP4 formats so that operators can trade a little precision for a large memory saving when that trade is worthwhile. NVFP4 is NVIDIA's 4-bit floating-point representation, tuned specifically for GPUs built on the Blackwell architecture; it compresses the memory footprint further while keeping any precision loss small enough to remain acceptable in most real workloads. The three speculative decoding methods each favor a different workload shape, so developers are free to mix and match them depending on the scenario they actually face in deployment, rather than being locked into a single fixed strategy.
Comparison with Competitors
Set against dense models of a similar parameter scale, Nemotron 3.5 Lightning gains clear advantages in both latency and throughput thanks to its MoE structure. Its 1M-token context also places it alongside the leading long-context models currently on the market, rather than trailing behind them by a wide margin. Perhaps the most telling signal is that SGLang finished the framework adaptation on the day of launch itself. That outcome reflects how tightly NVIDIA and the open-source community are now working together, and a day-zero cadence of this kind is normally reserved for flagship models, which is enough to show the high priority this release carries within the wider ecosystem that surrounds modern inference tooling and open model distribution today.
Industry Impact and Use Cases
Because it exposes an OpenAI-compatible calling interface, Nemotron 3.5 Lightning drops cleanly into existing agent workflows and supports localized, low-latency AI assistant scenarios without requiring heavy rework of application code. For teams that care about data privacy and tight cost control, the pairing of open weights with day-zero framework support lowers the barrier to bringing large-model capability into their own environment. It also gives smaller teams a realistic way to experiment with long-context inference that, until recently, would have been far too expensive to consider running on their own machines. For the open-model community, seeing day-zero support become routine would mark a meaningful step toward parity with closed labs that ship framework support on the same day by default. Seen from an industry standpoint, this kind of progress keeps lowering the barrier for both developers and everyday users, and the practical gains are arriving faster than many expected.