Phi

Phi-4

The Phi-4 model card says 14B parameters, dense decoder-only.

MIT · 14B · context 16,384

The Phi-4 model card says a 16k context. config.json max_position_embeddings is 16,384.

From the model card

Strengths

  • The Phi-4 card describes a 14B dense model aimed at complex reasoning, including math.
  • The card says the weights are MIT licensed and the model is text, primarily English.

Weaknesses

  • 16k context is short for long-document RAG.
  • The card says the model is primarily English.
  • No Frankenstein row yet.

Sources: Phi-4 model card · config.json

Memory at 8,192 tokens

Computed, not measured. Weights use llama.cpp’s whole-file averages (Q4_K_M = 4.8944 bits/weight, Q8_0 = 8.5008) on the total parameter count. KV is fp16 and does not depend on the quant. The formula, including the sliding-window and latent-KV cases, is on the methodology page.

Computed memory for Phi-4 at 8192 context
Q4_K_MQ8_0
Weights7.98 GiB13.85 GiB
KV at 8,1921.56 GiB1.56 GiB
Total9.54 GiB Computed15.42 GiB Computed

40 layers, 10 KV heads, head dim 128. Every layer stores the full context.

Tokens per second

Not yet measured. This lab has no Frankenstein row for Phi-4, so there is no estimated tok/s either.

Hardware tiers

Minimum is the smallest RAM that holds Q4_K_M plus 8k KV. Suggested does not name a second machine or a speed this lab has not run. Performance means a CUDA-class card with 12 GB or more of GDDR6 or GDDR6X and the weights fully on the card. The Tesla P40 is 24 GB of GDDR5, so a P40 speed is not the performance-tier number.

TierExample hardwareQuantFittok/s
MinimumCPU-only, with at least 11.54 GiB of system RAM for the Q4_K_M weights, 8k KV, and a 2 GiB planning allowance.Q4_K_MCPU-only, if the RAM is thereCPU: not yet measured
SuggestedA machine you are willing to wait on, or a GPU that can hold the quant. This lab has not measured a separate “comfortable” machine, so this row does not invent one.Q4_K_M until a Q8 run existsWhichever of the machines above actually holds the weightsnot yet measured
PerformanceA CUDA-class card with 12 GB or more of GDDR6 or GDDR6X, weights fully on the card.Q4_K_MFull VRAM on a 12 GB or larger GDDR6-class cardnot yet measured

Where it sits

Best-for is a judgment from the card’s stated jobs and the model’s size. It is not a speed ranking. The matching build is CPU coding box. Score it against a machine you have on What can my rig run?