What local AI models can the NVIDIA Tesla P100 PCIe 16GB run?
50 of our 115 models, up to gpt-oss-20b with long context. It is the largest current model that keeps room for long context on the Tesla P100 PCIe 16GB's 16 GB: at Q4 it fits with room for its full 128K context, at about 169 tok/s estimate. The largest that fits at all is Mistral-Small-3.2-24B-Instruct-2506 with room for only about 3K tokens of context (7K with 8-bit KV cache).
Memory type: HBM2 (source)
- Speed uses the Pascal constants fitted to one Tesla P40 point.
- year: no first-party launch date found.
- bus_bits: datasheet gives none.
- HBM card: speed constants were fitted on GDDR cards.
Specs from first-party pages (NVIDIA), accessed 2026-10-18. Speeds use the NVIDIA Pascal (CUDA) constants (how).
As an Amazon Associate I earn from qualifying purchases.
Every model, largest that fits first
| Model | Quant | File | Fits? | Speed estimate | Max context estimate |
|---|---|---|---|---|---|
| gpt-oss-20b | Q4_K_M | 11.67 GB | Fits | ~169 tok/s | 128K (full) |
| phi-4 | Q4_K_M | 9.05 GB | Fits | ~49 tok/s | 16K (full) |
| JetBrains Mellum2.1 12B-A2.5B Thinking | Q4_K_M | 8.07 GB | Fits | ~193 tok/s | 128K (full) |
| Gemma 4 12B | Q4_K_M | 7.12 GB | Fits | ~62 tok/s | 256K (full) |
| NVIDIA Nemotron 3.5 Bolt E4B | Q4_K_M | 6.18 GB | Fits | ~70 tok/s | 64K (full) |
| Qwen3.5-9B | Q4_K_M | 6.17 GB | Fits | ~60 tok/s | 256K (full) |
| Ternary Bonsai 2 27B | PTQ1_0 ternary | 5.95 GB | Fits | ~62 tok/s | 134K · 253K (8-bit) |
| Microsoft Fara1.5 9B | Q4_K_M | 5.91 GB | Fits | ~62 tok/s | 256K (full) |
| Xiaomi MiMo-V2.6-Distill-Qwen-9B | Q4_K_M | 5.84 GB | Fits | ~63 tok/s | 256K (full) |
| ZoomGain-9B | Q4_K_M | 5.63 GB | Fits | ~65 tok/s | 256K (full) |
| K2-Horizon-7B | Q4_K_M | 5.59 GB | Fits | ~77 tok/s | 62K · 117K (8-bit) |
| webAI TwIL-8B Math and code reasoning | Q4_K_M | 5.35 GB | Fits | ~80 tok/s | 57K · 108K (8-bit) |
| IBM Granite 4.2 8B | Q4_K_M | 5.35 GB | Fits | ~80 tok/s | 57K · 108K (8-bit) |
| PLaMo 3 Translate 8B Translation only | Q4_K_M | 5.28 GB | Fits | ~81 tok/s | 32K (full) |
| LFM2.5-8B-A1B | Q4_K_M | 5.16 GB | Fits | ~166 tok/s | 128K (full) |
| Gemma 4 E4B | Q4_K_M | 4.98 GB | Fits | ~86 tok/s | 128K (full) |
| Spark-X2.6-4B | Q4_K_M | 3.54 GB | Fits | ~116 tok/s | 300K · 568K (8-bit) |
| Gemma 4 E2B | Q4_K_M | 3.46 GB | Fits | ~118 tok/s | 128K (full) |
| Qwen3.5-4B | Q4_K_M | 3.01 GB | Fits | ~101 tok/s | 256K (full) |
| NVIDIA Nemotron 3 Nano 4B | Q4_K_M | 2.9 GB | Fits | ~103 tok/s | 256K (full) |
| Microsoft Fara1.5 4B | Q4_K_M | 2.88 GB | Fits | ~104 tok/s | 256K (full) |
| ZoomGain-4B | Q4_K_M | 2.71 GB | Fits | ~108 tok/s | 256K (full) |
| IBM Granite 4.2 3B | Q4_K_M | 2.24 GB | Fits | ~169 tok/s | 128K (full) |
| Raon-1 nano Early support | Q4_K_M | 2.08 GB | Weights fit context unknown | ~125 tok/s | Unknown (no KV data) |
| Ternary Bonsai 2 9B | PTQ1_0 ternary | 2 GB | Fits | ~128 tok/s | 256K (full) |
| PLaMo 3 Translate 2B Translation only | Q4_K_M | 1.81 GB | Fits | ~200 tok/s | 32K (full) |
| LFM3-1.7B | Q4_K_M | 1.1 GB | Fits | ~169 tok/s | 256K (full) |
| LFM3-VL-2B | Q4_K_M | 1.1 GB | Fits | ~169 tok/s | 128K (full) |
| Ternary Bonsai 2 4B | PTQ1_0 ternary | 0.95 GB | Fits | ~179 tok/s | 256K (full) |
| MiniCPM-V 4.7 1B | Q4_K_M | 0.67 GB | Fits | ~200 tok/s | 256K (full) |
| PLaMo 3 Translate 0.8B Translation only | Q4_K_M | 0.57 GB | Fits | ~417 tok/s | 32K (full) |
| BotPsy-0.8B | Q4_K_M | 0.53 GB | Fits | ~212 tok/s | 256K (full) |
| Ternary Bonsai 2 2B | PTQ1_0 ternary | 0.43 GB | Fits | ~222 tok/s | 256K (full) |
| LFM2-24B-A2B Older | Q4_K_M | 14.42 GB | Fits | ~151 tok/s | 27K · 52K (8-bit) |
| Qwen3-14B Older | Q4_K_M | 9 GB | Fits | ~50 tok/s | 35K · 40K (8-bit) |
| Qwen2.5-Coder-14B-Instruct Older | Q4_K_M | 8.99 GB | Fits | ~50 tok/s | 29K · 32K (8-bit) |
| DeepSeek-R1-Distill-Qwen-14B Older | Q4_K_M | 8.99 GB | Fits | ~50 tok/s | 29K · 56K (8-bit) |
| Mistral-Nemo-Instruct-2407 Older | Q4_K_M | 7.48 GB | Fits | ~59 tok/s | 44K · 84K (8-bit) |
| gemma-3-12b-it Older | Q4_K_M | 7.3 GB | Fits | ~60 tok/s | 107K · 128K (8-bit) |
| NVIDIA-Nemotron-Nano-9B-v2 Older | Q4_K_M | 6.53 GB | Fits | ~58 tok/s | 128K (full) |
| Qwen3-8B Older | Q4_K_M | 5.03 GB | Fits | ~85 tok/s | 40K (full) |
| Llama-3.1-8B-Instruct Older | Q4_K_M | 4.92 GB | Fits | ~87 tok/s | 75K · 128K (8-bit) |
| Qwen2.5-Coder-7B-Instruct Older | Q4_K_M | 4.68 GB | Fits | ~91 tok/s | 32K (full) |
| Mistral-7B-Instruct-v0.3 Older | Q4_K_M | 4.37 GB | Fits | ~96 tok/s | 32K (full) |
| Qwen3-4B Older | Q4_K_M | 2.5 GB | Fits | ~155 tok/s | 40K (full) |
| gemma-3-4b-it Older | Q4_K_M | 2.49 GB | Fits | ~155 tok/s | 128K (full) |
| Llama-3.2-3B-Instruct Older | Q4_K_M | 2.02 GB | Fits | ~184 tok/s | 110K · 128K (8-bit) |
| Qwen3-0.6B Older | Q4_K_M | 0.48 GB | Fits | ~453 tok/s | 40K (full) |
| Mistral-Small-3.2-24B-Instruct-2506 | Q4_K_M | 14.33 GB | Fits | ~32 tok/s | 3K · 7K (8-bit) Tight |
| Devstral-Small-2505 | Q4_K_M | 14.33 GB | Fits | ~32 tok/s | 3K · 7K (8-bit) Tight |
| LFM2.5-24B-A2B | Q4_K_M | 14.53 GB | No | — | — |
| gemma-3-27b-it Older | Q4_K_M | 16.55 GB | No | — | — |
| Kvark Sparkle 1.1 27B | Q4_K_M | 16.81 GB | No | — | — |
| Gemma 4 26B A4B | Q4_K_M | 17.04 GB | No | — | — |
| Qwen3.8-27B | Q4_K_M | 17.44 GB | No | — | — |
| Microsoft Fara1.5 27B | Q4_K_M | 17.53 GB | No | — | — |
| IBM Granite 4.2 30B | Q4_K_M | 17.72 GB | No | — | — |
| Qwen3.6-27B | Q4_K_M | 17.98 GB | No | — | — |
| Qwen3.5-27B | Q4_K_M | 17.98 GB | No | — | — |
| Meta Muse Glimmer 30B | Q4_K_M | 18.39 GB | No | — | — |
| GLM-4.7-Flash | Q4_K_M | 18.31 GB | No | — | — |
| Qwen3-Coder-30B-A3B-Instruct Older | Q4_K_M | 18.56 GB | No | — | — |
| Gemma 4 31B | Q4_K_M | 18.32 GB | No | — | — |
| Qwen3-30B-A3B Older | Q4_K_M | 18.63 GB | No | — | — |
| PLaMo 3 Translate 31B Translation only | Q4_K_M | 19.53 GB | No | — | — |
| Raon-1 mini preview Early support | Q4_K_M | 19.69 GB | No | — | — |
| Qwen3-32B Older | Q4_K_M | 19.76 GB | No | — | — |
| Qwen2.5-Coder-32B-Instruct Older | Q4_K_M | 19.85 GB | No | — | — |
| DeepSeek-R1-Distill-Qwen-32B Older | Q4_K_M | 19.85 GB | No | — | — |
| Xing4.0-29B-A4B | Q4_K_M | 20.06 GB | No | — | — |
| Nanbeige4.3-33B | Q4_K_M | 20.79 GB | No | — | — |
| MiniCPM-V 4.7 35B-A3B | Q4_K_M | 21.17 GB | No | — | — |
| K2-Horizon-32B | Q4_K_M | 21.08 GB | No | — | — |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB (2-bit) | native 2-bit | 22.02 GB | No | — | — |
| Qwen3.6-35B-A3B | Q4_K_M | 22.29 GB | No | — | — |
| Qwen3.5-35B-A3B | Q4_K_M | 22.29 GB | No | — | — |
| K2-Horizon-MoVA-36B-A4B | Q4_K_M | 22.37 GB | No | — | — |
| NVIDIA Nemotron 3.5 Lightning 30B-A3B | Q4_K_M | 25.48 GB | No | — | — |
| Recursive Flint 40B-A2.5B | Q4_1 | 25.99 GB | No | — | — |
| Ripple-65B Preview | Q4_K_M | 38.8 GB | No | — | — |
| Llama-3.3-70B-Instruct Older | Q4_K_M | 42.52 GB | No | — | — |
| DeepSeek-R1-Distill-Llama-70B Older | Q4_K_M | 42.52 GB | No | — | — |
| Qwen2.5-72B-Instruct Older | Q4_K_M | 47.42 GB | No | — | — |
| Qwen3-Coder-Next | Q4_K_M | 48.53 GB | No | — | — |
| Aleph Alpha Kolibri-1 78B-A3.5B | Q4_K_M | 49.76 GB | No | — | — |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B | UD-Q4_K_XL | 50.22 GB | No | — | — |
| gpt-oss-120b | Q4_K_M | 62.84 GB | No | — | — |
| NVIDIA Nemotron 3.5 Super VL 120B-A12B | Q4_K_M | 70.1 GB | No | — | — |
| Mistral Small 4 119B | Q4_K_M | 72.64 GB | No | — | — |
| GLM-4.5-Air Older | Q4_K_M | 73.5 GB | No | — | — |
| Qwen3.5-122B-A10B | Q4_K_M | 77.62 GB | No | — | — |
| Mistral Medium 3.5 128B | Q4_K_M | 78.41 GB | No | — | — |
| Ornith-1.5-122B | Q4_K_M | 78.92 GB | No | — | — |
| NVIDIA Nemotron 3 Super 120B-A12B Older | UD-Q4_K_M | 82.54 GB | No | — | — |
| Swift 1.5 Qwen3.8-Flash-Next | Q4_K_M | 119.6 GB | No | — | — |
| Qwen3.8-Flash-Next | Q4_K_M | 119.6 GB | No | — | — |
| StepFun Step 3.7 Flash | UD-Q4_K_M | 122.09 GB | No | — | — |
| Qwen3-235B-A22B | Q4_K_M | 142.15 GB | No | — | — |
| MiniMax-M3.1-Flash Unsloth 2-bit | UD-Q2_K_XL | 143.91 GB | No | — | — |
| DeepSeek-V4-Flash | UD-Q4_K_XL | 155.1 GB | No | — | — |
| MiniMax-M3.1-Flash Q2_K | Q2_K | 155.91 GB | No | — | — |
| Thinking Machines Inkling-Small Older | Q4_K_M | 159.34 GB | No | — | — |
| Xiaomi MiMo-V2.6-Flash 309B-A15B | MXFP4 | 167.37 GB | No | — | — |
| GLM-5.3-Flash | UD-Q4_K_XL | 199.71 GB | No | — | — |
| MiniMax-M3.1-Flash | Q4_K_M | 258.01 GB | No | — | — |
| Reflection Beam 1.0 501B-A23B | Q4_K_M | 302.41 GB | No | — | — |
| Thinking Machines Inkling 2 Small Early support | UD-Q4_K_XL | 354.15 GB | No | — | — |
| inclusionAI Ling-3.1-flash 560B-A25B | Q4_K_M | 357.97 GB | No | — | — |
| DeepSeek-V4.1-Flash (1017) | UD-Q4_K_XL | 500.71 GB | No | — | — |
| Meta Muse Spark Open 816B-A42B | UD-Q4_K_XL | 505.5 GB | No | — | — |
| Mistral Large 4 | UD-Q4_K_XL | 605 GB | No | — | — |
Q4_K_M unless the row says otherwise; UD- sizes are unsloth dynamic quants, the only Q4 build published for those models. Max context is an estimate of the longest context whose KV cache fits beside the model: default FP16 cache, then 8-bit (q8_0).
Speed calculator
estimateEstimate = bandwidth ÷ GB read per token (the model's listed GGUF file: Q4_K_M for most models, otherwise the quant named in the list; scaled by active/total parameters for MoE), scaled by a per-platform efficiency and fixed per-token costs calibrated to 45 measured llama.cpp benchmarks (2025–26). Fit = file size + 1.5 GB headroom ≤ total VRAM (on Macs, the part of unified memory macOS lets the GPU use by default), and room for at least 4K tokens of context with an 8-bit KV cache. For a model with no KV cache data, only the file and headroom are checked, so a fit says nothing about context. Multi-GPU: VRAM adds, speed stays one card's (a layer split runs the cards in turn); max context uses the total. Speed is for short context. Max context = what is left after the file and a 1 GB runtime reserve ÷ KV cache bytes per token, default FP16 and 8-bit (q8_0) cache. How speed works · how context works