AI news, decoded · 5 minutes

Nemotron 3.5 Bolt: NVIDIA's 74B Open Agent Model for 24GB GPUs

On October 15, 2026 (US time), NVIDIA released Nemotron 3.5 Bolt 74B-A9B right on schedule — one day before RTX Spark laptops went on sale on October 16. The weights went live on Hugging Face in the early hours of October 16 (Beijing time): the official repo nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16, plus official 24GB and GGUF versions in the same org, with community quantizations (unsloth, lmstudio-community) already following. It is a 74-billion-parameter mixture-of-experts (MoE) model that activates only about 9 billion parameters per inference, compressed via 2-bit quantization to run in 24GB of VRAM. The standout is the openness: fully open weights, training data, and training recipes under the OpenMDW-1.1 license, which explicitly permits commercial use; with day-one support for llama.cpp, vLLM, Ollama, Unsloth, and LM Studio, plus direct integration with the Hermes and OpenClaw agent frameworks. This is the most significant open-source release in the local AI space this year.

What happened

Start with the hard specs. The numbers behind Nemotron 3.5 Bolt 74B-A9B are straightforward: 74 billion total parameters (the model's knowledge base), but the MoE architecture activates only about 9 billion per inference — decoupling total knowledge from per-inference compute cost. Combined with 2-bit quantization and quantization-aware training (which simulates precision loss during training to preserve quality after compression), the whole model fits into 24GB. Architecture details are now public too: a Mamba-2 + Transformer hybrid MoE with a 256K context window; the 24GB version is natively 2-bit quantized and runs on laptops. NVIDIA's claim: faster than dense models of similar size, smarter than MoE models with a similar memory footprint.

The positioning is equally clear: this is built for agentic AI on RTX Spark. RTX Spark laptops pair a Grace CPU with a Blackwell GPU, up to 128GB of unified LPDDR5X memory (connected via NVLink-C2C), and 1 petaflop of AI compute, going on sale October 16. NVIDIA says local LLM downloads are already 5x all of last year — Bolt 74B is riding that wave: keeping 74B-class agents resident on personal computers to write code, monitor tasks, call tools, and run background jobs, all locally, without routing every request through a cloud API bill.

Two more things shipped in the same batch. First, PAIR (Personal AI Router) graduated from beta: a free, open-source tool that auto-discovers compatible PCs on a home network and distributes AI inference workloads across multiple machines. The full release adds llama.cpp support, direct model downloads from Hugging Face, and RTX Spark compatibility, running on Windows, Linux, and macOS. Second, the smaller Nemotron 3.5 Bolt E4B is dated October 23, targeting conversational agents and game NPCs — developers can already try it in SkyrimNET, the AI NPC modding platform for Skyrim.

Not just a model: a full local stack

What stands out is NVIDIA's playbook here: the model is the ammunition, the full local AI stack is the territory. The same software batch includes Llama for Windows (a free, open-source app built on llama.cpp that runs open models without touching the command line), MCP servers for NVIDIA's own apps (including the NVIDIA app, Broadcast, PAIR, RTX Remix, and Omniverse), Broadcast 2.3, and a sandboxed agent execution framework built with Microsoft (Microsoft Execution Containers) entering early preview. On the agent app side, GitHub Copilot, Hermes, OpenClaw, Perplexity, and others will streamline setup on RTX Spark with security features through OpenShell and MXC.

To translate: NVIDIA is answering the question every "local AI" launch of the past two years has dodged — how do you fit a frontier-class model into consumer hardware without stripping out what makes it frontier-class. Their answer: MoE decoupling knowledge from compute, plus 2-bit quantization-aware training, plus a full software stack for the model to live in. Whether that answer holds up now has a first independent data point: Artificial Analysis scores Bolt 21 on its Intelligence Index (v4.3.2), just 1.5 points behind NVIDIA's own 550B flagship, Nemotron 3 Ultra, at 22.9 — remarkably close for a model squeezed into 24GB. One catch: Bolt averages roughly 82,000 output tokens per task, nearly 3x the Ultra's ~28,000. Capable, but chatty.

One caveat

On quality there is now a first independent benchmark: Artificial Analysis scores Bolt 21 on its Intelligence Index (v4.3.2), just 1.5 points behind the 550B Nemotron 3 Ultra (22.9) — though Bolt averages ~82,000 output tokens per task, nearly 3x the Ultra. The post-quantization speed comparisons, however, still come from NVIDIA alone, and hosted speed and real-world cost remain unmeasured. By our own rules, that has to be stated up front: treat the score as a reference point, the speed claims as provisional.

Sources