Phi-3.5-MoE-instruct

41.9B parameters (2 of 16 experts per token), text in, text out, 128K tokens of context.

Verified 2026-10-25 Rank #358 in LLMs · score 81 method v1 Sources 3 Downloads 30d 67.1k · #364 of 619 tracked

Can I run it?25GB at INT4Estimatefits a 64 GB Mac (unified memory)method

Can I use it?MITPermissivelicence file2026-10-24

Do people use it?67.1kdownloads, 30 days#364 of 619 we track · below the median+3 likes, downloads −32% this week589 likesHugging Face2026-10-24

Worth reading

Picked by rule from 4 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Latest · 2 years ago

Show HN: RetNPhi – Phi-3.5 as a Byte-Level LM with RetNet-Inspired Efficiency

RetNPhi is an experimental architecture that transforms Phi-3.5 into a byte-level language model incorporating RetNet-inspired mechanisms.

The tokenizer defaults to a vocabulary of 256 bytes when no custom file path is provided.
The retention mechanism calculates decay factors using logarithmic scales between 1/32 and 1/512.

github.com 2024-09-02 2 HN pts 3.1k words about the Phi 3.5 family family

4 best of 4 tracked

The 1 pick above, plus these 3. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

ArticleMicrosoft releases powerful new Phi-3.5 models, beating Google, OpenAI and moreventurebeat.com · 2024-08-21 · 6 HN pts
ArticleRun with Phi 3.5-mini across GPUs, CPUs and OSessecondstate.io · 2024-09-03
ArticleLLM: Against all odds – The token paths not takenalgo-consult.com · 2025-11-21 · 1 HN pts

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 17 quantized builds and 12 Spaces.

As a service0no API provider listed
On own hardware17quantized builds
Built on129 fine-tunes · 3 adapters · 0 merges
In apps12Hugging Face Spaces

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 64 GB Mac (unified memory) at INT4

About 25 GB at INT4, 50 GB at FP8, 101 GB at BF16: weights only, so leave room for the KV cache. Every expert must be resident, so total parameters count.

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

3 dated articles and 14 repo commits so far

Teal is the weights (2024-08-17), amber triangles are 3 dated articles (each links to it), grey ticks are repo commits.

What people built on it

29 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

29 derivatives on the Hub, most of them quantized

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

GGUF bartowski/Phi-3.5-MoE-instruct-GGUF for llama.cpp, Ollama, LM Studio 40.8k downloads

All variants →

Reference

Everything else, folded: the full tables, every source.

Benchmarks

14 rows

Benchmarks 14 rows

measured third-party leaderboard rowauthor-reported number from the model card or eval files (14)No confidence intervals unless the source gave them; settings differ between rows, so compare within a benchmark, not across. Date = the benchmark's own date when it gives one, else when we read it.

Benchmark	Value	Setting / notes	Reported by	Date
author-reported ARC Challenge · Chat and follow instructions	91	—	model author	2026-10-07
author-reported Arena-Hard · Chat and follow instructions	37.9	—	model author	2026-10-07
author-reported BBH · Reason through hard problems	79.1	—	model author	2026-10-07
author-reported Commonsense QA sets · Chat and follow instructions	84.6	—	model author	2026-10-07
author-reported Commonsense QA sets · Chat and follow instructions	89.6	—	model author	2026-10-07
author-reported Commonsense QA sets · Chat and follow instructions	88.6	—	model author	2026-10-07
author-reported Commonsense QA sets · Chat and follow instructions	78	—	model author	2026-10-07
author-reported GPQA Diamond · Reason through hard problems	36.8	—	model author	2026-10-07
author-reported HellaSwag · Chat and follow instructions	83.8	—	model author	2026-10-07
author-reported MMLU · Reason through hard problems	78.9	—	model author	2026-10-07
author-reported MMLU-Pro · Reason through hard problems	54.3	—	model author	2026-10-07
author-reported MMMLU · Work across languages	69.9	—	model author	2026-10-07
author-reported TruthfulQA · Chat and follow instructions	77.5	—	model author	2026-10-07
author-reported Winogrande · Chat and follow instructions	81.3	—	model author	2026-10-07

Use-case evidence

5 use cases

What it is good at 5 use cases

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

24 fields

Listed in LLMs · MoE

Facts

Phi-3.5-MoE-instruct has 41.9B parameters. (safetensors metadata, observed 2026-10-24)
Phi-3.5-MoE-instruct is a mixture-of-experts model with 16 experts, routing 2 per token. (config.json, observed 2026-10-24)
Context length is 131,072 tokens (128K). (config.json max_position_embeddings, observed 2026-10-24)
License: MIT (friction: none). Permissive. (license file, observed 2026-10-24)
Created 2024-08-17, last modified 2025-12-10. (HF API, observed 2026-10-24)

Pipeline tag	text-generation
Library	transformers
Architectures	PhiMoEForCausalLM
Model type	phimoe
Dtype	bfloat16
Params by dtype	BF16: 41,873,153,344
Size band	XL
MoE	yes
Experts	16
Experts per token	2
Context length	131,072 (config.json max_position_embeddings)
Languages	multilingual
Papers	arXiv:2404.14219, arXiv:2407.13833, arXiv:2403.06412
License	MIT (mit)
Hub snapshot	2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · PhiMoEForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · PhiMoEForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for Phi-3.5-MoE-instruct: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)	101 GB · Multi-GPU node (<640 GB)
FP8 (1 byte/param)	50 GB · Single high-end GPU (<80 GB)
INT4 (0.5 bytes/param)	25 GB · Single high-end GPU (<80 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra. MoE: all experts must be resident, so total parameters count.

All variants

29 variants

Lineage

Original model Family: microsoft/Phi-3.5 29 derivatives on the Hub

quantized: 17finetune: 9adapter: 3

Top variants (29 shown, ranked by downloads)

Variant	Relation	Quant	Params	Band	License	Author	Likes	Downloads
bartowski/Phi-3.5-MoE-instruct-GGUF	quantized	GGUF	—	—	mit	bartowski	15	40.8k
second-state/Phi-3.5-MoE-instruct-GGUF	quantized	GGUF	—	—	mit	second-state	0	8.7k
lmstudio-community/Phi-3.5-MoE-instruct-GGUF	quantized	GGUF	—	—	mit	lmstudio-community	6	7.7k
mradermacher/Phi-3.5-MoE-instruct-GGUF	quantized	GGUF	—	—	mit	mradermacher	0	4k
gaianet/Phi-3.5-MoE-instruct-GGUF	quantized	GGUF	—	—	mit	gaianet	0	1.9k
tensorblock/Phi-3.5-MoE-instruct-GGUF	quantized	GGUF	—	—	mit	tensorblock	0	1.8k
inference-optimization/Phi-3.5-MoE-0.8B-A0.2B	finetune	—	803M	S	mit	inference-optimization	1	1.5k
AbteeXAILab/lumynax-frontier-phi-35-moe-instruct-gguf	quantized	gguf	—	—	mit	AbteeXAILab	0	899
MikeKuykendall/phi-3.5-moe-q4-k-m-cpu-offload-gguf	quantized	gguf	—	—	mit	MikeKuykendall	0	799
rockon1095/Phi-3.5-MoE-instruct-Q4_0-GGUF	quantized	Q4_0	—	—	mit	rockon1095	0	625
FullnameNameUser/Phi-3.5-MoE-instruct-Q4_K_M-GGUF	quantized	GGUF	—	—	mit	FullnameNameUser	0	379
kylesayrs/Phi-3.5-MoE-0.8B-A0.2B	finetune	—	803M	S	mit	kylesayrs	2	188
sheldonrobinson/Phi-3.5-MoE-instruct-Q4_K_M-GGUF	quantized	GGUF	—	—	mit	sheldonrobinson	0	224
noctrex/Phi-3.5-MoE-instruct-MXFP4_MOE-GGUF	quantized	GGUF	—	—	—	noctrex	0	223
MikeKuykendall/phi-3.5-moe-cpu-offload-gguf	quantized	gguf	—	—	mit	MikeKuykendall	0	209
MikeKuykendall/phi-3.5-moe-q8-0-cpu-offload-gguf	quantized	gguf	—	—	mit	MikeKuykendall	0	199
sheldonrobinson/Phi-3.5-MoE-instruct-Q4_0-GGUF	quantized	Q4_0	—	—	mit	sheldonrobinson	0	191
MikeKuykendall/phi-3.5-moe-q2-k-cpu-offload-gguf	quantized	gguf	—	—	mit	MikeKuykendall	0	186
Claw123454321/CartoonCraft	finetune	—	—	—	apache-2.0	Claw123454321	3	0
xmindai/xm-moe-medical	adapter	—	—	—	—	xmindai	0	82
Kris2017/Phi-3.5-MoE-instruct-W4A16-AutoRound	quantized	-W4A16	5.83B	M	mit	Kris2017	0	70
BLOZZT/FISICO	adapter	—	—	—	osl-3.0	BLOZZT	1	0
inference-optimization/Phi-3.5-MoE-1.0B-A0.7B-small	finetune	—	976M	S	mit	inference-optimization	0	48
fufuya/test	finetune	—	—	—	creativeml-openrail-m	fufuya	0	0
Aggic/Aggic	adapter	—	—	—	mit	Aggic	0	0
Ligteller/London_time	finetune	—	—	—	apache-2.0	Ligteller	0	0
SipsaLabs/phi-3.5-moe-instruct-streaming-bpw5	finetune	—	—	—	other	SipsaLabs	0	0
SipsaLabs/phi-3.5-moe-uc-v3-bpw5	finetune	—	—	—	other	SipsaLabs	0	0
SipsaLabs/phi-3.5-moe-instruct-uc-v3-bpw5	finetune	—	—	—	other	SipsaLabs	0	0

Repository changelog

14 commits

Repository changelog 14 commits

Date	Commit	SHA
2025-12-10	Update README.md (#57)	4368845
2025-12-10	Upload data_summary_card.md (#56)	b1544d1
2025-03-07	Update README.md (#52)	c5ec144
2024-10-24	Update README.md	96bbd87
2024-10-24	Update README.md	4d50ff9
2024-09-27	Update README.md	a722adb
2024-08-30	Update README.md	ae6cb90
2024-08-30	Update README.md. Added Korean benchmarks. (#38)	a09cede
2024-08-21	Update README.md	482a9ba
2024-08-20	Update README.md	c36d6a2
2024-08-20	Update README.md	8f3232c
2024-08-20	upload model card	46fb142
2024-08-17	upload initial files	aa13f6c
2024-08-17	initial commit	1f005e0

Articles and coverage

4 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Analysis3

Microsoft releases powerful new Phi-3.5 models, beating Google, OpenAI and more

venturebeat.com2024-08-21about the Phi 3.5 familyHN 6 pts · 2 comments

Show HN: RetNPhi – Phi-3.5 as a Byte-Level LM with RetNet-Inspired Efficiency

github.com2024-09-02authorityabout the Phi 3.5 familyHN 2 pts · 0 comments3,121 words

RetNPhi is an experimental architecture that transforms Phi-3.5 into a byte-level language model incorporating RetNet-inspired mechanisms. The 578-line implementation, dated Aug 28, 2024, enables the model to process raw byte sequences, allowing for universal file type handling. It supports dual-mode processing for efficient training and inference.written, reviewed, cited

The tokenizer defaults to a vocabulary of 256 bytes when no custom file path is provided.
The retention mechanism calculates decay factors using logarithmic scales between 1/32 and 1/512.
Recurrent mode initializes a cache state with a fixed shape of 32 tokens, 96 heads, and 96 dimensions.

LLM: Against all odds – The token paths not taken

algo-consult.com2025-11-21about the Phi 3.5 familyHN 1 pts · 0 comments223,753 words

All coverage tracked for this model (4) — append-only, nothing is ever removed

Title	Source	Published	First seen	Signals
LLM: Against all odds – The token paths not taken	algo-consult.com	2025-11-21	2026-10-07	hn
Run with Phi 3.5-mini across GPUs, CPUs and OSes	secondstate.io	2024-09-03	2026-10-07	hn
Show HN: RetNPhi – Phi-3.5 as a Byte-Level LM with RetNet-Inspired Efficiency	github.com	2024-09-02	2026-10-07	hn
Microsoft releases powerful new Phi-3.5 models, beating Google, OpenAI and more	venturebeat.com	2024-08-21	2026-10-07	hn

Papers

3 papers

Papers 3

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone this model

Marah Abdin, Jyoti Aneja, Hany Awadalla +126 · 2024-04-22 · PDF · HF paper page (262 upvotes)
Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cycle this model

Emman Haider, Daniel Perez-Becker, Thomas Portet +28 · 2024-07-18 · PDF · HF paper page (12 upvotes)
CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean cited in card

Eunsu Kim, Juyoung Suk, Philhoon Oh +3 · 2024-03-11 · PDF · HF paper page (3 upvotes)

Similar models

3 models

Similar models

GLM-4.7-Flash · 31.2B
also MoE, same task (text-generation), more providers, permissive license
Kimi-Linear-48B-A3B-Instruct · 49.1B
same size band (XL), also MoE, same task (text-generation), more providers, permissive license
Qwen3-Coder-30B-A3B-Instruct · 30.5B
also MoE, same task (text-generation), more providers, permissive license

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:38 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:46 UTC · Summary: 2026-10-07 15:09 UTC · Last verified 2026-10-25

Follow Phi-3.5-MoE-instruct

Watch this model

/feeds/models/microsoft--Phi-3.5-MoE-instruct.xml(RSS 2.0) · every change to Phi-3.5-MoE-instruct
/feeds/families/microsoft--Phi-3.5.xml(RSS 2.0) · every change across the Phi-3.5 family

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.