Home › Method

Hardware fit ESTIMATE

A first-order answer to "what does it take to load this?" It is weights only. It says nothing about throughput, and it is wrong for your batch size, context length, and serving stack, which add KV cache and activations on top.

Formula

memory_gb = params_total × bytes_per_param × 1.2 / 1e9

params_total	Sum of tensor sizes from the repo's safetensors metadata (shown in Facts as "Params by dtype"). For MoE models this is all experts: they must all be resident, so the active-parameter count does not reduce memory.
bytes_per_param	BF16/FP16: 2 · FP8: 1 · INT4: 0.5. The three rows on a model page are the same weights at each precision, regardless of what the lab shipped.
1.2 overhead	Flat 20% for framework buffers, embeddings kept in higher precision, and fragmentation. Not tuned per model.

Classes

Class	Weights memory	Typical hardware
edge	< 4 GB	Phones, laptops without a discrete GPU, small single-board computers
single-gpu	< 24 GB	One consumer GPU (RTX 3090/4090-class) or a 24 GB cloud GPU
single-high-end-gpu	< 80 GB	One A100/H100/H200/RTX PRO 6000-class GPU
multi-gpu	< 640 GB	One node with up to 8 × 80 GB GPUs
cluster	≥ 640 GB	More than one node; expert or tensor parallelism across machines

The class on a model page is computed at BF16; where the engine also emits FP8 and INT4 classes they are shown beside each row.

Worked example

A 27.8B dense model: 27.8e9 × 2 bytes × 1.2 = 66.7 GB at BF16 (single high-end GPU), 33.4 GB at FP8 (single high-end GPU with headroom for a long KV cache), 16.7 GB at INT4 (single consumer GPU). A 2.78T MoE: 6.67 TB at BF16, which is a cluster at any precision.

Not included

KV cache: scales with context length × batch × layers × heads; often larger than the weights at long context.
Activations, CUDA graphs, and compiled kernels.
Quantization quality: INT4 memory is shown even when no INT4 checkpoint exists or when the lab's own quant is the only sensible one.