Draft, because the machine notes are still incomplete. The speeds below are from the run. They are also in the benchmark table.
Frankenstein, 18 Nov 2026, about 39 minutes from 4:42 to 5:21 PM Chicago time. Two Xeon E5-2697 v2 CPUs, 12 cores per socket, 24 physical cores and 48 threads. Two NUMA nodes. 288 GB installed DDR3 registered ECC (279 GiB usable), all 24 slots filled, running at 1067 MT/s because each channel has three DIMMs. Tesla P40, 24 GB, driver 580.178.04. Ubuntu, Linux 7.0.0-43-generic. Ollama 0.35.0, not restarted for the suite. No standalone llama.cpp CLI on the box.
The build page is CPU-only used Xeon. Prices are still not pulled. Wall watts were not metered.
How the numbers were taken
Ollama’s HTTP /api/generate, stream=false, think=false, temperature=0, seed=42, num_predict=128. Every cell is three runs. Generation tok/s is the median of eval_count / eval_duration. All four models are Qwen3 at Q4_K_M.
Short prompt: about 110–118 tokens, num_ctx=2048. Long prompt: about 4,012–4,020 tokens, num_ctx=8192.
Prompt tok/s in the table is the first uncached pass. After that, the same text hits Ollama’s prefix cache and comes back at 10k–170k tok/s. That is not a prefill rate, and it is not in the table. Where the first pass was already cached, the cell is n/a.
OLLAMA_NUM_PARALLEL=2 often left the previous model loaded, so some RSS figures include a leftover model. Where a clean reading exists, the table uses it and the note says so.
frank-qwen3-30b-64k is the Qwen3-30B-A3B weight blob (30.5B total, about 3B active) with a Modelfile num_ctx of 65536. This run used 2048 and 8192. 64k was not loaded.
What the CPU actually did
Default thread count, num_gpu=0.
| Model | Context | Gen tok/s | Uncached prefill |
|---|---|---|---|
| qwen3 4B instruct | 2048 | 12.36 | n/a |
| qwen3 4B instruct | 8192 | 6.62 | ~52.6 |
| qwen3 8B | 2048 | 6.91 | n/a |
| qwen3 8B | 8192 | 4.66 | 31.3 |
| qwen3 14B | 2048 | 4.41 | n/a |
| qwen3 14B | 8192 | 3.02 | 17.1 |
| 30B MoE (A3B) | 2048 | 9.58 | n/a |
| 30B MoE (A3B) | 8192 | 8.73 | 53.2 |
The 30B MoE is the one I would actually leave loaded. On CPU it beat the dense 8B and the dense 14B at both context lengths. Short, it was a bit slower than the 4B (9.58 vs 12.36). Long, it was faster than the 4B (8.73 vs 6.62). Prefill at about 4k tokens was 53 tok/s, ahead of the 8B (31) and the 14B (17).
That is not a dense 30B. A dense model that size was not in this run. The win is the MoE: most of the weights sit in the 288 GB, and only about 3B of them work on each token.
Do not use all 48 threads
Same 14B, short prompt, CPU only.
| Threads | Median gen tok/s |
|---|---|
| Ollama default | 4.41 |
| 24 (physical cores) | 4.24 |
| 48 (all SMT) | 1.33 |
The default already tracks the physical cores. Forcing all 48 threads was about 3.2× slower (1.33 vs 4.24). On this board I will leave num_thread alone, or set 24. I will not set 48.
Long context
From 2048 to 8192, the dense 4B almost halved (12.36 to 6.62). The dense 14B dropped by about a third (4.41 to 3.02). The 30B MoE moved from 9.58 to 8.73. The long prompt hurts the dense models more. 64k was not tested.
The P40
Same models, num_gpu=99, same quant, same prompts. Generation tok/s, median of three:
| Model | 2048 | 8192 | Versus CPU |
|---|---|---|---|
| 4B | 69.57 | 60.03 | 5.6× / 9.1× |
| 8B | 44.18 | 40.22 | 6.4× / 8.6× |
| 14B | 25.42 | 21.85 | 5.8× / 7.2× |
| 30B MoE | 65.29 | 57.64 | 6.8× / 6.6× |
Call it 6–9× the CPU generation rate. The 30B MoE at 8192 used 18.9 GiB of the P40’s 24 GiB, with the card otherwise clear of leftover models. The 14B settled at 11.1 GiB once the previous model was gone. A 64k context for the MoE was not loaded. From the 8k VRAM, most of that 18.9 GiB is weights, so 65536 is likely to spill. That is a guess about fit, not a measurement.
GPU prefill that was actually uncached: 4B long about 1100 (calibration, load may overlap), 8B long 748, 14B long 434, 30B MoE long 871. The short-prompt GPU prefills in the table are first-run approximations, not cached medians.
The RAM is slow on purpose, right now
Every channel has three DIMMs, so the box runs at 1067 MT/s even though the sticks are rated 1333, 1600, or 1866. These CPU numbers are the machine as it is built. They are not the ceiling of the E5-2697 v2 if the DIMMs were ever rearranged to two per channel.
Still open
- Wall watts, idle and under one of these runs.
- Whether 65536 context for the 30B MoE fits on the P40.
- A dense 30B, if I ever have a reason to time one.
- Noise, and where this chassis is allowed to live.
- A price for the parts, with the day it was pulled. Nothing on the build page is a price yet.
No run failed. Every cell produced 128 tokens. The model that was merely slow was the 14B with 48 threads, and that was a setting, not a broken quant.