granite-4.0-h-tiny

ibm-granite Apache 2.0 out 13 months ago

6.94B parameters (6 of 64 experts per token), text in, text out, 128K tokens of context.

Verified 2026-10-25 Rank #631 in LLMs · score 73.3 method v1 Sources 3 Downloads 30d 32.3k · #416 of 619 tracked

Can I run it?4.2GB at INT4Estimatefits a 16 GB cardmethod

Can I use it?Apache 2.0Permissive; patent grantlicence file2026-10-24

Do people use it?32.3kdownloads, 30 days#416 of 619 we track · below the median+3 likes, downloads −25% this week214 likesHugging Face2026-10-24

Worth reading

Picked by rule from 4 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Latest · 13 months ago

IBM Granite 4.0

IBM launches Granite 4.0, a new family of open language models featuring a hybrid Mamba/transformer architecture designed for enterprise efficiency.

Granite 4.0-H-Small exceeds all open weight models on IFEval except Llama 4 Maverick, a 402B-parameter model over 12 times its size.
Granite 4.0 models are compatible with AMD Instinct MI-300X GPUs, enabling even further reduction of their memory footprint.

ibm.com 2025-10-02 10 HN pts · 3 comments 2.7k words about the granite 4.0 family family

4 best of 4 tracked

The 1 pick above, plus these 3. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

ArticleGranite 4.0 3B Vision: Compact Multimodal Intelligence for Enterprise Documentshuggingface.co · 2026-04-01
ArticleIBM Granite 4.0 Tiny Previewibm.com · 2025-05-05 · 4 HN pts · official
ArticleGranite 4.0 Nano: Just how small can you go?huggingface.co · 2025-10-28

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 38 quantized builds and 3 Spaces.

As a service0no API provider listed
On own hardware38quantized builds
Built on88 fine-tunes · 0 adapters · 0 merges
In apps3Hugging Face Spaces

Top Spaces running it: TobDeBer/Granite4Family ♥ 4

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 16 GB card at INT4

About 4.2 GB at INT4, 8.3 GB at FP8, 17 GB at BF16: weights only, so leave room for the KV cache. Every expert must be resident, so total parameters count.

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

2 dated articles and 8 repo commits so far

Teal is the weights (2025-09-16), amber triangles are 2 dated articles (each links to it), grey ticks are repo commits.

What people built on it

46 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

46 derivatives on the Hub, most of them quantized

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

GGUF unsloth/granite-4.0-h-tiny-GGUF for llama.cpp, Ollama, LM Studio 284.3k downloads MLX lmstudio-community/granite-4.0-h-tiny-MLX-8bit for Apple silicon 9.4k downloads AWQ cyankiwi/granite-4.0-h-tiny-AWQ-4bit for vLLM, SGLang 174.3k downloads

All variants →

Reference

Everything else, folded: the full tables, every source.

Benchmarks

8 rows

Benchmarks 8 rows

measured third-party leaderboard rowauthor-reported number from the model card or eval files (8)No confidence intervals unless the source gave them; settings differ between rows, so compare within a benchmark, not across. Date = the benchmark's own date when it gives one, else when we read it.

Benchmark	Value	Setting / notes	Reported by	Date
author-reported AGI EVAL	62.15	0-shot, CoT	model author	2026-10-07
author-reported AlpacaEval · Chat and follow instructions	30.61	—	model author	2026-10-07
author-reported BBH · Reason through hard problems	66.34	3-shot, CoT	model author	2026-10-07
author-reported GPQA Diamond · Reason through hard problems	32.59	0-shot, CoT	model author	2026-10-07
author-reported IFEval · Chat and follow instructions	84.78	Instruct, Strict	model author	2026-10-07
author-reported IFEval · Chat and follow instructions	78.1	Prompt, Strict	model author	2026-10-07
author-reported MMLU · Reason through hard problems	68.65	5-shot	model author	2026-10-07
author-reported MMLU-Pro · Reason through hard problems	44.94	5-shot, CoT	model author	2026-10-07

Use-case evidence

6 use cases

What it is good at 6 use cases

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

22 fields

Listed in LLMs · MoE

Facts

granite-4.0-h-tiny has 6.94B parameters. (safetensors metadata, observed 2026-10-24)
granite-4.0-h-tiny is a mixture-of-experts model with 64 experts, routing 6 per token. (config.json, observed 2026-10-24)
Context length is 131,072 tokens (128K). (config.json max_position_embeddings, observed 2026-10-24)
License: Apache 2.0 (friction: none). Permissive; patent grant. (license file, observed 2026-10-24)
Created 2025-09-16, last modified 2025-11-03. (HF API, observed 2026-10-24)

Pipeline tag	text-generation
Library	transformers
Architectures	GraniteMoeHybridForCausalLM
Model type	granitemoehybrid
Dtype	bfloat16
Params by dtype	BF16: 6,939,037,248
Size band	M
MoE	yes
Experts	64
Experts per token	6
Context length	131,072 (config.json max_position_embeddings)
Papers	arXiv:0000.00000
License	Apache 2.0 (apache-2.0)
Hub snapshot	2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · GraniteMoeHybridForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · GraniteMoeHybridForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for granite-4.0-h-tiny: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)	17 GB · Single GPU (<24 GB)
FP8 (1 byte/param)	8.3 GB · Single GPU (<24 GB)
INT4 (0.5 bytes/param)	4.2 GB · Single GPU (<24 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra. MoE: all experts must be resident, so total parameters count.

All variants

40 variants

Lineage

Original model Family: ibm-granite/granite-4.0 46 derivatives on the Hub

quantized: 38finetune: 8

Top variants (40 shown, ranked by downloads)

Variant	Relation	Quant	Params	Band	License	Author	Likes	Downloads
unsloth/granite-4.0-h-tiny-GGUF	quantized	GGUF	—	—	apache-2.0	unsloth	40	284.3k
cyankiwi/granite-4.0-h-tiny-AWQ-4bit	quantized	AWQ	2.00B	S	apache-2.0	cyankiwi	3	174.3k
lmstudio-community/granite-4.0-h-tiny-GGUF	quantized	GGUF	—	—	apache-2.0	lmstudio-community	3	154.8k
ibm-granite/granite-4.0-h-tiny-GGUF	quantized	GGUF	—	—	apache-2.0	ibm-granite	29	62.9k
bartowski/ibm-granite_granite-4.0-h-tiny-GGUF	quantized	GGUF	—	—	—	bartowski	6	40.9k
unsloth/granite-4.0-h-tiny	finetune	—	6.94B	M	apache-2.0	unsloth	4	26.3k
RedHatAI/granite-4.0-h-tiny-FP8-dynamic	quantized	-FP8	6.95B	M	apache-2.0	RedHatAI	3	18.9k
lmstudio-community/granite-4.0-h-tiny-MLX-8bit	quantized	MLX	6.94B	M	apache-2.0	lmstudio-community	1	9.4k
mradermacher/granite-4.0-h-tiny-i1-GGUF	quantized	GGUF	—	—	apache-2.0	mradermacher	0	9.4k
mlx-community/Granite-4.0-H-Tiny-4bit-DWQ	quantized	-4bit	6.94B	M	apache-2.0	mlx-community	5	3.3k
mradermacher/granite-4.0-h-tiny-GGUF	quantized	GGUF	—	—	apache-2.0	mradermacher	0	2.9k
lmstudio-community/granite-4.0-h-tiny-MLX-4bit	quantized	MLX	6.94B	M	apache-2.0	lmstudio-community	0	2.6k
lmstudio-community/granite-4.0-h-tiny-MLX-6bit	quantized	MLX	6.94B	M	apache-2.0	lmstudio-community	0	2.5k
unsloth/granite-4.0-h-tiny-FP8-Dynamic	quantized	-FP8	6.95B	M	apache-2.0	unsloth	1	2.5k
tatsuyaaaaaaa/granite-4.0-h-tiny-gguf	quantized	gguf	—	—	apache-2.0	tatsuyaaaaaaa	0	2.5k
Myric/granite-4.0-h-tiny-APEX-GGUF	quantized	GGUF	—	—	apache-2.0	Myric	2	1.9k
DevQuasar/ibm-granite.granite-4.0-h-tiny-GGUF	quantized	GGUF	—	—	—	DevQuasar	0	1.8k
lmstudio-community/granite-4.0-h-tiny-MLX-5bit	quantized	MLX	6.94B	M	apache-2.0	lmstudio-community	0	1.3k
mlx-community/granite-4.0-h-tiny-3bit-MLX	quantized	MLX	6.94B	M	apache-2.0	mlx-community	2	1.2k
MichielBuisman/granite-4.0-h-tiny-dutch-calibrated-gguf	quantized	gguf	—	—	apache-2.0	MichielBuisman	1	1.1k
cyankiwi/granite-4.0-h-tiny-AWQ-8bit	quantized	AWQ	2.75B	M	apache-2.0	cyankiwi	0	1k
taronaeo/Granite-4.0-H-Tiny-BE-GGUF	quantized	GGUF	—	—	apache-2.0	taronaeo	0	787
noctrex/Granite-4.0-H-Tiny-MXFP4_MOE-GGUF	quantized	GGUF	—	—	—	noctrex	1	686
mlx-community/granite-4.0-h-tiny-5bit-MLX	quantized	MLX	6.94B	M	apache-2.0	mlx-community	2	631
fedora-copr/granite-4.0-h-tiny-quantized.w8a8	quantized	-quant	6.95B	M	apache-2.0	fedora-copr	0	692
pszemraj/granite-4.0-h-7b-heretic	finetune	—	6.94B	M	apache-2.0	pszemraj	7	324
mlx-community/granite-4.0-h-tiny-8bit	quantized	-8bit	6.94B	M	apache-2.0	mlx-community	0	635
mlx-community/granite-4.0-h-tiny-6bit-MLX	quantized	MLX	6.94B	M	apache-2.0	mlx-community	1	569
amd/granite-4.0-h-tiny-w8a8-llmcompressor	quantized	-w8a8	6.94B	M	apache-2.0	amd	0	434
RepublicOfKorokke/granite-4.0-h-tiny-mlx-mxfp4	quantized	mlx	6.94B	M	apache-2.0	RepublicOfKorokke	0	411
AmpereComputing/granite-4.0-h-tiny-GGUF	quantized	GGUF	—	—	—	AmpereComputing	0	389
Open4bits/granite-4.0-h-tiny-mlx-fp16	finetune	mlx	6.94B	M	apache-2.0	Open4bits	1	329
introvoyz041/granite-4.0-h-tiny-mlx-4Bit	quantized	mlx	6.94B	M	apache-2.0	introvoyz041	0	241
sizzlebop/granite-4.0-h-tiny-IQ4_NL-GGUF	quantized	GGUF	—	—	apache-2.0	sizzlebop	1	185
Open4bits/granite-4.0-h-tiny-f16	finetune	—	6.94B	M	apache-2.0	Open4bits	1	160
sizzlebop/granite-4.0-h-tiny-Q8_0-GGUF	quantized	Q8_0	—	—	apache-2.0	sizzlebop	0	195
glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think	finetune	—	—	—	apache-2.0	glogwa68	2	94
KavinduHansaka/granite-4.0-h-tiny-gguf	quantized	gguf	—	—	mit	KavinduHansaka	0	189
ethicalabs/granite-4.0-h-tiny-MLX	quantized	MLX	6.94B	M	apache-2.0	ethicalabs	0	179
guignome/granite-4.0-h-tiny-FP8-Dynamic	quantized	-FP8	7.09B	M	—	guignome	0	146

Repository changelog

8 commits

changed in the last 7 daysNew on the watch list: 2026-10-23all changes

Repository changelog 8 commits

Date	Commit	SHA
2025-11-03	model card update	791e0d3
2025-11-03	Add MOT badge (#6)	d281c3e
2025-10-07	adding default system prompt	bd583fd
2025-10-01	model card update	31c096d
2025-09-30	model card update	351a64d
2025-09-24	model card upload	b8fb3be
2025-09-22	model upload	a6188e1
2025-09-16	initial commit	cf39883

Articles and coverage

4 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Official2

IBM Granite 4.0

ibm.com2025-10-02officialauthorityabout the granite 4.0 familyHN 10 pts · 3 comments2,656 words

IBM launches Granite 4.0, a new family of open language models featuring a hybrid Mamba/transformer architecture designed for enterprise efficiency. Released under an Apache 2.0 license, these models are the first open models to receive ISO 42001 certification and offer over 70% reduction in RAM for long inputs. The hybrid architecture combines Mamba-2 layers with transformer attention, and IBM offers an uncapped indemnity for third party IP claims alongside a bug bounty program for security flaws.written, reviewed, cited

Granite 4.0-H-Small exceeds all open weight models on IFEval except Llama 4 Maverick, a 402B-parameter model over 12 times its size.
Granite 4.0 models are compatible with AMD Instinct MI-300X GPUs, enabling even further reduction of their memory footprint.

IBM Granite 4.0 Tiny Preview

ibm.com2025-05-05officialauthorityabout the granite 4.0 familyHN 4 pts · 0 comments1,604 words

IBM Granite 4.0 Tiny Preview is a preliminary open source model available under an Apache 2.0 license. It utilizes a hybrid Mamba-2/Transformer architecture with 7B total parameters and 1B active parameters at inference time. The model is pending support in Hugging Face transformers and vLLM before official local platform partner availability.written, reviewed, cited

Granite 4.0 Tiny Preview runs several concurrent 128K context sessions on consumer GPUs under $350 USD.
The model has only seen 2.5T of a planned 15T or more training tokens but rivals Granite 3.3 2B Instruct.

All coverage tracked for this model (4) — append-only, nothing is ever removed

Title	Source	Published	First seen	Signals
Granite 4.0 3B Vision: Compact Multimodal Intelligence for Enterprise Documents	huggingface.co	2026-04-01	2026-10-07	hn
Granite 4.0 Nano: Just how small can you go?	huggingface.co	2025-10-28	2026-10-07	hn
IBM Granite 4.0	ibm.com official	2025-10-02	2026-10-07	hn
IBM Granite 4.0 Tiny Preview	ibm.com official	2025-05-05	2026-10-07	hn

Similar models

3 models

Similar models

OLMoE-1B-7B-0924 · 6.92B
same size band (M), also MoE, same task (text-generation), permissive license
LFM2.5-8B-A1B · 8.47B
same size band (M), also MoE, same task (text-generation)
Phi-tiny-MoE-instruct · 3.76B
same size band (M), also MoE, same task (text-generation), permissive license

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:42 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:46 UTC · Summary: 2026-10-07 15:13 UTC · Last verified 2026-10-25

Follow granite-4.0-h-tiny

Watch this model

/feeds/models/ibm-granite--granite-4.0-h-tiny.xml(RSS 2.0) · every change to granite-4.0-h-tiny
/feeds/families/ibm-granite--granite-4.0.xml(RSS 2.0) · every change across the granite-4.0 family

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.