NVIDIA Releases Nemotron 3.5 Bolt Multimodal MoE Model

The model's performance was evaluated using a rigorous suite of multimodal and reasoning benchmarks. While specific MMLU and HumanEval scores for the Bolt variant are being integrated into the technical report, the model's architecture is optimized to match the reasoning capabilities of the 120B Super VL variant in specific agentic workflows. The evaluation methodology focused on real-world agentic task completion, visual grounding accuracy, and instruction following in multilingual contexts.

• Efficiency: Higher throughput than equivalent dense models
• Intelligence: Outperforms similar-footprint MoE competitors
• Optimization: Native 2-bit quantization variant for 24GB VRAM
• Evaluation Focus: Agentic task reliability and visual reasoning

Deployment & Quantization

One of the most critical aspects for developers is the model's accessibility on local hardware. NVIDIA has released a BF16 full-precision version intended for researchers and developers who wish to perform further customization or post-training. However, for the majority of the deployment community, the 24GB builds are the primary focus.

The native 2-bit quantization variant is a game-changer, allowing the 74B model to fit comfortably within 24GB of memory. This enables the model to run alongside other applications on high-end consumer GPUs. NVIDIA provides optimized builds specifically for llama.cpp (GGUF format) and vLLM, ensuring seamless integration into existing inference pipelines.

• Full Precision: BF16 for customization and post-training
• Edge Optimized: 2-bit quantization for 24GB VRAM
• Inference Engines: Support for llama.cpp (GGUF) and vLLM
• Hardware Target: RTX-class GPUs and local workstations

Use Cases

Nemotron 3.5 Bolt is not just a chatbot; it is an agentic engine. Its ability to 'see' and 'reason' makes it ideal for complex workflows where visual input is required to drive actions. For example, in automated coding environments, it can analyze UI screenshots to debug frontend issues or suggest layout improvements.

Other high-value use cases include intelligent RAG (Retrieval-Augmented Generation) systems that incorporate visual documents, local robotics control where latency is critical, and multilingual customer support agents that can process visual evidence (like photos of broken products) in real-time.

• Agentic Tasks: Visual-based tool use and decision making
• Coding: UI/UX analysis and visual debugging
• RAG: Multimodal document retrieval and reasoning
• Robotics: Low-latency local visual processing

Getting Started

Developers can access Nemotron 3.5 Bolt through several channels. The full-precision weights are available for download for those looking to fine-tune the model. For immediate inference, the GGUF and vLLM-compatible builds are the recommended starting point.

As part of the GTC Berlin portfolio, NVIDIA also released companion models, including the Nemotron Retriever embedding models, which can be used in conjunction with Bolt to build complete, end-to-end multimodal RAG pipelines. Check the NVIDIA Open Model Hub for the latest weights and deployment scripts.

• Weights: Available via NVIDIA Open Model Hub
• Formats: GGUF (llama.cpp) and vLLM
• Ecosystem: Compatible with Nemotron Retriever embedding models

Sources