NVIDIA expands its Nemotron family with a 30‑billion‑parameter model that boosts agentic AI performance, paired with a new open‑source routing library to streamline model selection.

NVIDIA has unveiled Nemotron 3.5 Lightning, a 30‑billion‑parameter language model designed to accelerate agentic AI workloads, alongside NeMo Switchyard, an open‑source routing library that simplifies dynamic model selection across heterogeneous GPU clusters.

Nemotron 3.5 Lightning: A Bigger, Faster Model

Building on the Nemotron 3 series, the Lightning variant adds roughly 10 billion parameters, bringing the total to 30 billion. The extra capacity is paired with NVIDIA’s latest RTX 6000 Ada GPU optimizations, delivering up to a 2× speedup on inference for complex reasoning tasks while consuming less power per token.

The model is pretrained on a diverse mix of publicly available text and code, enabling it to handle both natural‑language prompts and tool‑use instructions. Early benchmarks show improved success rates in multi‑step planning scenarios, a core requirement for autonomous agents.

NeMo Switchyard: Routing Intelligence for Multi‑Model Pipelines

NeMo Switchyard introduces a flexible routing layer that can dispatch incoming requests to the most appropriate model version in real time. By analyzing request characteristics—such as token length, latency constraints, and required tool usage—the library selects between Nemotron 3.5 Lightning, smaller Nemotron variants, or even specialized vision‑language models.

Switchyard’s design is hardware‑agnostic, supporting NVIDIA DGX, RTX workstations, and cloud‑based GPU instances. It also integrates with NVIDIA’s TensorRT‑LLM runtime, ensuring that selected models run at peak efficiency.

Benefits for Agentic AI Deployments

The combination of a larger, more capable model and a smart routing engine addresses two long‑standing challenges in agentic AI: latency and resource waste. Developers can now deploy a single endpoint that automatically balances load, reducing the need for manual model orchestration.

  • Lower inference latency for complex tasks
  • Dynamic scaling across GPU clusters
  • Reduced operational overhead for model management

These improvements are especially relevant for applications such as autonomous assistants, real‑time data analysis, and AI‑driven robotics, where timely and accurate decision‑making is critical.

NeMo Switchyard lets us treat model selection as a first‑class service, freeing engineers to focus on higher‑level agent logic.

NVIDIA plans to make both Nemotron 3.5 Lightning and NeMo Switchyard available under its open‑source AI ecosystem, encouraging community contributions and broader adoption across industries.

For the full announcement, see NVIDIA blog post on Nemotron Lightning and Switchyard.