Comparison
Models
Strengths and weaknesses are paraphrased from each model card, and the card is linked. Memory is computed from the parameter count and the published attention config. A tokens-per-second figure is labeled measured only when it is a Frankenstein row. Everything else says not yet measured. There are no prices on this page.
Strengths and weaknesses are paraphrased from the linked model cards. Parameter counts and context lengths are the figures those cards publish. Architecture fields are from the linked config.json and are used only for the KV formula. This file contains no tokens-per-second figures and no prices. Measured speeds are joined from benchmarks.json at build time. The formula is on the methodology page. To score one of these against a machine you actually have, use What can my rig run?
| License | Strengths | Weaknesses | Best for | tok/s | |||||
|---|---|---|---|---|---|---|---|---|---|
| Qwen3-4B | 4B | Apache-2.0 | 32,768 (YaRN 131,072) | The Qwen3 card describes one model with a thinking mode and a non-thinking mode. | This repository is a text model. Vision is a different Qwen checkpoint. | Cheap 8B chat | 3.4 GiB Computed | 5.1 GiB Computed | CPU 6.62 at 8192, measured · P40 60.03 at 8192, measured (GDDR5) · GDDR6 12 GB+ not yet measured |
| Qwen3-8B | 8.2B | Apache-2.0 | 32,768 (YaRN 131,072) | The Qwen3 card describes one model with a thinking mode and a non-thinking mode. | This repository is a text model. Vision is a different Qwen checkpoint. | Cheap 8B chat | 5.8 GiB Computed | 9.2 GiB Computed | CPU 4.66 at 8192, measured · P40 40.22 at 8192, measured (GDDR5) · GDDR6 12 GB+ not yet measured |
| Qwen3-14B | 14.8B | Apache-2.0 | 32,768 (YaRN 131,072) | The Qwen3 card describes one model with a thinking mode and a non-thinking mode. | This repository is a text model. Vision is a different Qwen checkpoint. | CPU coding box | 9.7 GiB Computed | 15.9 GiB Computed | CPU 3.02 at 8192, measured · P40 21.85 at 8192, measured (GDDR5) · GDDR6 12 GB+ not yet measured |
| Qwen3-30B-A3B | 30.5B total, 3.3B active | Apache-2.0 | 32,768 (YaRN 131,072) | Mixture-of-experts: the card says 3.3B parameters are active per token, out of 30.5B total. | Memory follows the 30.5B total. All experts are stored. The 3.3B figure is compute, not disk. | CPU-only used Xeon | 18.1 GiB Computed | 30.9 GiB Computed | CPU 8.73 at 8192, measured · P40 57.64 at 8192, measured (GDDR5) · GDDR6 12 GB+ not yet measured |
| Qwen3-32B | 32.8B | Apache-2.0 | 32,768 (YaRN 131,072) | The Qwen3 card describes one model with a thinking mode and a non-thinking mode. | Dense 32.8B: unlike the 30B-A3B model, every parameter is active, and the weight file is the full count. | CPU-only used Xeon | 20.7 GiB Computed | 34.5 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| Llama 3.1 8B Instruct | 8B | Llama 3.1 Community License | 131,072 | The Llama 3.1 card describes an instruct model for dialogue, with tool use and a 128k context. | The license requires attribution and has a monthly-active-user clause. It is not Apache-2.0. | Family server | 5.6 GiB Computed | 8.9 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| Llama 3.3 70B Instruct | 70B | Llama 3.3 Community License | 131,072 | The Llama 3.3 card describes a 70B instruct model with tool use and a 128k context. | Dense 70B. The weight file is the full count, and a 24 GB card does not hold Q4_K_M plus 8k KV under the formula on this site. | CPU-only used Xeon | 42.4 GiB Computed | 71.8 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| Gemma 3 4B | 4B | Gemma terms | 131,072 | The Gemma 3 card describes a multimodal model: text and image in, text out. | The license is the Gemma terms and the prohibited-use policy, not Apache-2.0. | Home assistant and RAG | 2.5 GiB Computed | 4.2 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| Gemma 3 12B | 12B | Gemma terms | 131,072 | The Gemma 3 card describes a multimodal model: text and image in, text out. | The license is the Gemma terms and the prohibited-use policy, not Apache-2.0. | Home assistant and RAG | 7.6 GiB Computed | 12.7 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| Gemma 3 27B | 27B | Gemma terms | 131,072 | The Gemma 3 card describes a multimodal model: text and image in, text out. | The license is the Gemma terms and the prohibited-use policy, not Apache-2.0. | Used 24 GB GPU | 16.4 GiB Computed | 27.8 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| Phi-4 | 14B | MIT | 16,384 | The Phi-4 card describes a 14B dense model aimed at complex reasoning, including math. | 16k context is short for long-document RAG. | CPU coding box | 9.5 GiB Computed | 15.4 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| Mixtral 8x7B InstructRetired | 46.7B total, 12.9B active | Apache-2.0 | 32,768 | The announcement describes a sparse mixture of experts: 8 experts, 2 active per token, Apache-2.0. | Mistral's docs say this model was retired on 30 March 2025. The listed replacement is Mistral Small. | CPU-only used Xeon | 27.6 GiB Computed | 47.2 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| Mistral Small 4 | 119B total, 6.5B active | Apache-2.0 | 262,144 | The docs describe a multimodal model (text and image) with tool use, a 256k context, and Apache-2.0 weights. | Weight memory follows the 119B total, not the active count. | CPU-only used Xeon | 68.0 GiB Computed | 117.9 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| DeepSeek-R1-Distill-Qwen-7B | 7B | MIT, plus the Qwen base license (Apache-2.0) | 131,072 | The DeepSeek-R1 card says these dense models are distilled from DeepSeek-R1's reasoning traces, and are the practical way to run that behavior without the 671B model. | Reasoning traces spend context and time. A short answer is not what this model is distilled to do. | Cheap 8B chat | 4.4 GiB Computed | 7.4 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| DeepSeek-R1-Distill-Qwen-14B | 14B | MIT, plus the Qwen base license (Apache-2.0) | 131,072 | The DeepSeek-R1 card says these dense models are distilled from DeepSeek-R1's reasoning traces. | Reasoning traces spend context and time. | CPU coding box | 9.5 GiB Computed | 15.4 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| DeepSeek-R1-Distill-Qwen-32B | 32B | MIT, plus the Qwen base license (Apache-2.0) | 131,072 | The DeepSeek-R1 card says these dense models are distilled from DeepSeek-R1's reasoning traces. | Reasoning traces spend context and time. Dense 32B weights follow the full count. | CPU-only used Xeon | 20.2 GiB Computed | 33.7 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| DeepSeek-R1-Distill-Llama-8B | 8B | MIT, plus the Llama 3.1 Community License | 131,072 | The DeepSeek-R1 card says these dense models are distilled from DeepSeek-R1's reasoning traces. | Two licenses apply: MIT on DeepSeek's files, and the Llama 3.1 Community License on the base. | Cheap 8B chat | 5.6 GiB Computed | 8.9 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| DeepSeek-R1-Distill-Llama-70B | 70B | MIT, plus the Llama 3.3 Community License | 131,072 | The DeepSeek-R1 card says these dense models are distilled from DeepSeek-R1's reasoning traces. | Dense 70B. A 24 GB card does not hold Q4_K_M plus 8k KV under the formula on this site. | CPU-only used Xeon | 42.4 GiB Computed | 71.8 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| gpt-oss-20b | 21B total, 3.6B active | Apache-2.0, plus the OpenAI usage policy | 131,072 | OpenAI's model card describes a text model with configurable reasoning effort and agentic tool use, under Apache-2.0. | OpenAI says the model will not work correctly without the harmony response format. | Used 24 GB GPU | 12.2 GiB Computed | 21.0 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
| gpt-oss-120b | 117B total, 5.1B active | Apache-2.0, plus the OpenAI usage policy | 131,072 | OpenAI's model card describes a text model with configurable reasoning effort and agentic tool use, under Apache-2.0. | OpenAI says the model will not work correctly without the harmony response format. | CPU-only used Xeon | 67.0 GiB Computed | 116.1 GiB Computed | CPU not yet measured · GDDR6 12 GB+ not yet measured |
Scroll sideways for the memory and speed columns. Q4_K_M and Q8_0 use llama.cpp’s published bits-per-weight averages. KV is fp16 at 8,192 tokens, with sliding-window and latent-KV rules where the config calls for them.
No model matches that filter.