Qwen3-30B-A3B
The Qwen3-30B-A3B model card says 30.5B total parameters and 3.3B activated.
The model card says 32,768 tokens natively and 131,072 with YaRN. config.json max_position_embeddings is 40,960. The Frankenstein tag sets num_ctx 65536; the measured runs used 2048 and 8192.
From the model card
Strengths
- Mixture-of-experts: the card says 3.3B parameters are active per token, out of 30.5B total.
- The same Qwen3 card describes thinking and non-thinking modes, and presents the family as a step up on reasoning, math, coding, and agent tasks.
Weaknesses
- Memory follows the 30.5B total. All experts are stored. The 3.3B figure is compute, not disk.
- This repository is a text model.
- Context past 32,768 needs YaRN. The measured runs did not load a 65,536 context.
Sources: Qwen3-30B-A3B model card · Qwen3 GitHub · config.json
Memory at 8,192 tokens
Computed, not measured. Weights use llama.cpp’s whole-file averages (Q4_K_M = 4.8944 bits/weight, Q8_0 = 8.5008) on the total parameter count. KV is fp16 and does not depend on the quant. The formula, including the sliding-window and latent-KV cases, is on the methodology page.
| Q4_K_M | Q8_0 | |
|---|---|---|
| Weights | 17.38 GiB | 30.18 GiB |
| KV at 8,192 | 0.75 GiB | 0.75 GiB |
| Total | 18.13 GiB Computed | 30.93 GiB Computed |
48 layers, 4 KV heads, head dim 128. Every layer stores the full context.
Tokens per second
These are the Frankenstein rows whose model name matches this card. Generation tok/s is the median of three runs. Prompt tok/s is uncached prefill, or n/a when that pass was not captured. Thread-sweep rows are included and labeled in the notes. They are not the headline number in the tiers below.
| Device | Quant | Context | Gen tok/s | Prompt tok/s | Date |
|---|---|---|---|---|---|
| CPU | Q4_K_M | 2,048 | 9.58 Measured | n/a | 2026-11-18 |
| CPU | Q4_K_M | 8,192 | 8.73 Measured | 53.2 | 2026-11-18 |
| Tesla P40 (GDDR5) | Q4_K_M | 2,048 | 65.29 Measured | 255 | 2026-11-18 |
| Tesla P40 (GDDR5) | Q4_K_M | 8,192 | 57.64 Measured | 871 | 2026-11-18 |
The full notes, including RSS and leftover models, are on the benchmark table.
Hardware tiers
Minimum is the smallest RAM that holds Q4_K_M plus 8k KV. Suggested does not name a second machine or a speed this lab has not run. Performance means a CUDA-class card with 12 GB or more of GDDR6 or GDDR6X and the weights fully on the card. The Tesla P40 is 24 GB of GDDR5, so a P40 speed is not the performance-tier number.
| Tier | Example hardware | Quant | Fit | tok/s |
|---|---|---|---|---|
| Minimum | CPU-only, with at least 20.13 GiB of system RAM for the Q4_K_M weights, 8k KV, and a 2 GiB planning allowance. | Q4_K_M | CPU-only, if the RAM is there | CPU: 8.73 gen tok/s, 53.2 prompt tok/s, context 8,192, 2026-11-18, measured |
| Suggested | A machine you are willing to wait on, or a GPU that can hold the quant. This lab has not measured a separate “comfortable” machine, so this row does not invent one. | Q4_K_M until a Q8 run exists | Whichever of the machines above actually holds the weights | not yet measured |
| Performance | A CUDA-class card with enough GDDR6 or GDDR6X to hold about 19.13 GiB (Q4_K_M weights, 8k KV, and a 1 GiB planning allowance). 12 GB is not enough for that total. | Q4_K_M | Full VRAM only if the card is larger than 12 GB | not yet measured |
Measured on this lab’s Tesla P40, which is not the performance tier: 57.64 gen tok/s at context 8,192, Q4_K_M, 2026-11-18. GDDR5, 24 GB.
Where it sits
Best-for is a judgment from the card’s stated jobs and the model’s size. It is not a speed ranking. The matching build is CPU-only used Xeon. Score it against a machine you have on What can my rig run?