What GPU do you need to run NVIDIA Nemotron 3.5 Bolt E4B?

8B parameters (Hugging Face, accessed 2026-10-23). Q4_K_M GGUF: 6.18 GB (source). Architecture config (Hugging Face config.json): 4 global attention layers × 2 KV heads × 512 head dim, plus 20 sliding-window layers (512-token window); the last 18 layers reuse earlier layers' KV and store none of their own, so about 16 KB of FP16 KV cache per token of context (how we estimate context).

Text, image and audio in. Built on Gemma 4 E4B (Gemma4ForConditionalGeneration): "E4B" means about 4B effective parameters, and the 8.0B count is Hugging Face's safetensors total of 7,996,156,490, which includes the vision and audio encoders. Q4_K_M is NVIDIA's official GGUF from their "4GB" repo (Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf, 6,180,446,208 bytes), so despite the repo name the file is 6.18 GB. Unsloth's UD quants are smaller; every size is in the table below. NVIDIA pitches the 4GB build for local assistants and tool use (https://x.com/i/status/2113657112408785153); Unsloth says it runs on 6 GB of RAM (https://x.com/i/status/2113658964365971681). Those are their claims, not checked here. Fit ignores the vision projector (mmproj-F16 990,373,568 bytes, mmproj-Q8_0 559,875,776 bytes), which is extra if you send images. Release date is the Hugging Face repo creation date of the BF16 weights.