Gemma 3

Gemma 3 4B

Google's Gemma 3 model card lists a 4B size. This page uses that size class, not a separate counted total.

Gemma terms · 4B · context 131,072

The Gemma 3 model card says 128k context for the 4B, 12B, and 27B models.

From the model card

Strengths

  • The Gemma 3 card describes a multimodal model: text and image in, text out.
  • The card says 128k context and coverage of 140+ languages.

Weaknesses

  • The license is the Gemma terms and the prohibited-use policy, not Apache-2.0.
  • The text KV formula does not include the vision encoder.
  • No Frankenstein row yet.

Sources: Gemma 3 model card · config.json (public mirror; official weights are gated)

Memory at 8,192 tokens

Computed, not measured. Weights use llama.cpp’s whole-file averages (Q4_K_M = 4.8944 bits/weight, Q8_0 = 8.5008) on the total parameter count. KV is fp16 and does not depend on the quant. The formula, including the sliding-window and latent-KV cases, is on the methodology page.

Computed memory for Gemma 3 4B at 8192 context
Q4_K_MQ8_0
Weights2.28 GiB3.96 GiB
KV at 8,1920.27 GiB0.27 GiB
Total2.55 GiB Computed4.23 GiB Computed

34 layers, 4 KV heads, head dim 256. Sliding window 1024. A layer is full attention when (layer index + 1) is divisible by 6; the others use the window.

Tokens per second

Not yet measured. This lab has no Frankenstein row for Gemma 3 4B, so there is no estimated tok/s either.

Hardware tiers

Minimum is the smallest RAM that holds Q4_K_M plus 8k KV. Suggested does not name a second machine or a speed this lab has not run. Performance means a CUDA-class card with 12 GB or more of GDDR6 or GDDR6X and the weights fully on the card. The Tesla P40 is 24 GB of GDDR5, so a P40 speed is not the performance-tier number.

TierExample hardwareQuantFittok/s
MinimumCPU-only, with at least 4.55 GiB of system RAM for the Q4_K_M weights, 8k KV, and a 2 GiB planning allowance.Q4_K_MCPU-only, if the RAM is thereCPU: not yet measured
SuggestedA machine you are willing to wait on, or a GPU that can hold the quant. This lab has not measured a separate “comfortable” machine, so this row does not invent one.Q4_K_M until a Q8 run existsWhichever of the machines above actually holds the weightsnot yet measured
PerformanceA CUDA-class card with 12 GB or more of GDDR6 or GDDR6X, weights fully on the card.Q4_K_MFull VRAM on a 12 GB or larger GDDR6-class cardnot yet measured

Where it sits

Best-for is a judgment from the card’s stated jobs and the model’s size. It is not a speed ranking. The matching build is Home assistant and RAG. Score it against a machine you have on What can my rig run?