Model Follower.

Up to date

Rebuilt
2026-10-25 06:54 UTC
Release radar
117 on watch
Hub snapshot
2026-10-25
Re-checked
daily · every number sourced and dated

Models › Qwen1.5 family

Qwen1.5-MoE-A2.7B

Hugging Face ↗ Follow changes

Qwen tongyi-qianwen out 3 years ago

14.3B parameters, 2.7B active (4 of 60 experts per token), text in, text out, 8K tokens of context.

Verified 2026-10-25 Rank #536 in LLMs · score 75.3 method v1 Sources 3 Downloads 30d 89.1k · #326 of 619 tracked

Can I run it?8.6GB at INT4Estimatefits a 16 GB cardmethod
Can I use it?tongyi-qianwenCustom license; read the repo LICENSE before commercial hostinglicence file2026-10-24
Do people use it?89.1kdownloads, 30 days#326 of 619 we track · below the median+2 likes, downloads −27% this week236 likesHugging Face2026-10-24

Worth reading

Picked by rule from 7 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Latest · 3 years ago

Qwen1.5-Moe: Matching 7B Model Performance with 1/3 Activated Parameters

Qwen1.5-MoE-A2.7B matches 7B model performance with only 2.7 billion activated parameters using a 64-expert configuration.

  • The model partitions a single FFN into segments to generate 64 experts without increasing the total parameter count.
  • Training from scratch is inefficient, so the team repurposes the existing Qwen-1.8B model to initialize Qwen1.5-MoE-A2.7B.
qwenlm.github.io 2024-03-28 104 HN pts · 10 comments 1.5k words about the Qwen1.5 family family

7 best of 7 tracked

The 1 pick above, plus these 6. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

  1. ArticleQwen1.5-110Bqwenlm.github.io · 2024-04-26 · official
  2. ArticleQwen1.5-32B: Fitting the Capstone of the Qwen1.5 Language Model Seriesqwenlm.github.io · 2024-04-05 · official
  3. ArticleQwen1.5qwenlm.github.io · 2024-02-05 · 5 HN pts · official
  4. DiscussionQwen1.5-110B Previewtwitter.com · 2024-04-20
  5. ArticleComplete tutorial: Run Qwen LLM across your local machines without Python/C++secondstate.io · 2024-02-22
  6. ArticleRun Alicloud's Qwen LLM with a 2MB cross GPU/CPU rust appsecondstate.io · 2024-02-28

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 12 quantized builds and 3 Spaces.

  • As a service0no API provider listed
  • On own hardware12quantized builds
  • Built on5232 fine-tunes · 20 adapters · 0 merges
  • In apps3Hugging Face Spaces

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 16 GB card at INT4

About 8.6 GB at INT4, 17 GB at FP8, 34 GB at BF16: weights only, so leave room for the KV cache. Every expert must be resident, so total parameters count.

16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 8.6 GB of weights (estimate; KV cache and activations extra)INT48.6 GBFP8: about 17 GB of weights (estimate; KV cache and activations extra)FP817 GBBF16: about 34 GB of weights (estimate; KV cache and activations extra)BF1634 GB
16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 8.6 GB of weights (estimate; KV cache and activations extra)INT48.6 GBFP8: about 17 GB of weights (estimate; KV cache and activations extra)FP817 GBBF16: about 34 GB of weights (estimate; KV cache and activations extra)BF1634 GB

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

2 dated articles and 19 repo commits so far

Teal is the weights (2024-02-29), amber triangles are 2 dated articles (each links to it), grey ticks are repo commits.

MarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2024-04-18: 1 repo commit2024-04-04: 1 repo commit2024-03-29: 1 repo commit2024-03-28: 8 repo commits2024-03-21: 1 repo commit2024-03-15: 2 repo commits2024-03-14: 1 repo commit2024-03-04: 1 repo commit2024-02-29: 3 repo commitsQwen1.5-Moe: Matching 7B Model Performance with 1/3 Activated Parameters · 2024-03-28Qwen1.5 · 2024-02-05Weights 2024-02-29 · 2024-02-29Weights 2024-02-29now
MarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2024-04-18: 1 repo commit2024-04-04: 1 repo commit2024-03-29: 1 repo commit2024-03-28: 8 repo commits2024-03-21: 1 repo commit2024-03-15: 2 repo commits2024-03-14: 1 repo commit2024-03-04: 1 repo commit2024-02-29: 3 repo commitsQwen1.5-Moe: Matching 7B Model Performance with 1/3 Activated Parameters · 2024-03-28Qwen1.5 · 2024-02-05Weights 2024-02-29 · 2024-02-29Weights 2024-02-29now

What people built on it

64 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

64 derivatives on the Hub, most of them fine-tunes

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

12 quantized (Hugging Face model tree)32 fine-tunes (Hugging Face model tree)20 adapters (Hugging Face model tree)12 quantized32 fine-tunes20 adapters
12 quantized (Hugging Face model tree)32 fine-tunes (Hugging Face model tree)20 adapters (Hugging Face model tree)12 quantized32 fine-tunes20 adapters
GGUF tensorblock/Qwen1.5-MoE-A2.7B-GGUF for llama.cpp, Ollama, LM Studio 6k downloads

All variants →

Reference

Everything else, folded: the full tables, every source.

What the card says

1 cited sentence

Summary generated, cited

Qwen1.5-MoE-A2.7B is a transformer-based Mixture of Experts decoder-only language model for text generation.[1]

  1. “transformer-based MoE decoder-only language model”

Generated 2026-10-07 15:12 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

1 use case

What it is good at 1 use case

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

24 fields

Listed in LLMs · MoE

Facts

Pipeline tagtext-generation
Librarytransformers
ArchitecturesQwen2MoeForCausalLM
Model typeqwen2_moe
Dtypebfloat16
Params by dtypeBF16: 14,315,784,192
Size bandL
MoEyes
Experts60
Experts per token4
Context length8,192 (config.json max_position_embeddings)
Languagesen
Licensetongyi-qianwen (other)
Hub snapshot2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · Qwen2MoeForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · Qwen2MoeForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for Qwen1.5-MoE-A2.7B: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)34 GB · Single high-end GPU (<80 GB)
FP8 (1 byte/param)17 GB · Single GPU (<24 GB)
INT4 (0.5 bytes/param)8.6 GB · Single GPU (<24 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra. MoE: all experts must be resident, so total parameters count.

All variants

40 variants

Lineage

Original model Family: Qwen/Qwen1.5 64 derivatives on the Hub

quantized: 12adapter: 20finetune: 32

Top variants (40 shown, ranked by downloads)

Relation
Quant
Size
40 rows
Variant Relation Quant Params Band License Author Likes Downloads
tensorblock/Qwen1.5-MoE-A2.7B-GGUF quantized GGUF — — other tensorblock 2 6k
QuantFactory/Qwen1.5-MoE-A2.7B-Wikihow-GGUF quantized GGUF — — apache-2.0 QuantFactory 1 5.7k
ducanhdinh/Qwen1.5-MoE-A2.7B-Finetuning adapter — — — apache-2.0 ducanhdinh 1 4.3k
ducanhdinh/Qwen1.5-MoE-A2.7B-MidAlign adapter — — — apache-2.0 ducanhdinh 2 2.6k
sai-lakkshmii/Qwen1.5-MoE-A2.7B-squad-lora-latest adapter — — — other sai-lakkshmii 0 1.9k
Kieran2828/Qwen1.5-MoE-A2.7B-Q4_K_M-GGUF quantized GGUF — — other Kieran2828 0 1.3k
MaziyarPanahi/Qwen1.5-MoE-A2.7B-Wikihow finetune — 14.3B L apache-2.0 MaziyarPanahi 4 843
TULLUS/Qwen1.5-MoE-A2.7B finetune — 14.3B L other TULLUS 0 989
liodon-ai/Qwen1.5-MoE-A2.7B-imatrix-GGUF quantized imatrix — — other liodon-ai 0 927
TULLUS/Qwen1.5-MoE-A2.7B-GGUF quantized GGUF — — other TULLUS 1 758
ktila8/Qwen1.5-MoE-A2.7B-Q4_K_M-GGUF quantized GGUF — — other ktila8 0 720
xd2010/Qwen1.5-MOE-sft-math7k-densemixer finetune — 14.3B L — xd2010 0 687
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-1e-3-gamma finetune — 14.3B L — HectorHe 1 546
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-1e-3-gamma-1epoch finetune — 14.3B L — HectorHe 0 593
ducanhdinh/Qwen1.5-MoE-A2.7B-SQuAD-Task-Only adapter — — — apache-2.0 ducanhdinh 0 499
ducanhdinh/Qwen1.5-MoE-A2.7B-Finetuning-lb0.01 adapter — — — apache-2.0 ducanhdinh 0 452
zeromodels/qwen1.5-moe-a2.7b finetune — — — other zeromodels 0 439
cApMaN1987/Qwen1.5-MoE-A2.7B-Q4_K_M-GGUF quantized GGUF — — other cApMaN1987 0 392
CryptikTech/Qwen1.5-MoE-A2.7B-Q4_K_M-GGUF quantized GGUF — — other CryptikTech 0 378
xd2010/Qwen1.5-MOE-aux-free-sft-math7k-1e-3-gamma-1epoch finetune — 14.3B L — xd2010 0 367
xd2010/Qwen1.5-MOE-aux-free-sft-math7k-1e-3-gamma-1epo finetune — 14.3B L — xd2010 0 328
wtfisupdennys/Qwen1.5-MoE-A2.7B-Q4_K_M-GGUF quantized GGUF — — other wtfisupdennys 0 322
Kieran2828/Qwen1.5-MoE-A2.7B-Q8_0-GGUF quantized Q8_0 — — other Kieran2828 0 285
xd2010/Qwen1.5-MOE-sft-math7k-sft-2epochs-frozen-router finetune — 14.3B L — xd2010 0 282
HectorHe/Qwen1.5-MOE-sft-nemotron-code finetune — 14.3B L — HectorHe 1 180
yschoi31/Qwen1.5-MoE-A2.7B-Eagle3 finetune — 142M S apache-2.0 yschoi31 0 228
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-1e-4-gamma finetune — 14.3B L — HectorHe 1 162
ducanhdinh/Qwen1.5-MoE-A2.7B-SNLI-Task-Only adapter — — — apache-2.0 ducanhdinh 0 198
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-5e-5-gamma finetune — 14.3B L — HectorHe 1 144
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-1e-2-gamma finetune — 14.3B L — HectorHe 1 135
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-remov-aux-only finetune — 14.3B L — HectorHe 1 125
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-1e-6-gamma finetune — 14.3B L — HectorHe 1 99
HectorHe/Qwen1.5-MOE-sft-coommonsense15k finetune — 14.3B L — HectorHe 1 97
HectorHe/Qwen1.5-MOE-sft-math7k-sft-epoch1 finetune — 14.3B L — HectorHe 1 95
HectorHe/Qwen1.5-MOE-sft-math7k-sfttest finetune — 14.3B L — HectorHe 0 114
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-1e-4-gamma-3epoch finetune — 14.3B L — HectorHe 0 114
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-5e-3-gamma finetune — 14.3B L — HectorHe 0 102
MoeReward/reward_lora_qwen_1_5_base adapter — — — — MoeReward 0 95
HectorHe/Qwen1.5-MOE-sft-coommonsense15k-aux-free-1e-5 finetune — 14.3B L — HectorHe 0 94
HectorHe/Qwen1.5-MOE-aux-free-sft-math7k-3e-3-gamma finetune — 14.3B L — HectorHe 0 91

Repository changelog

19 commits

Repository changelog 19 commits

DateCommitSHA
2024-04-18 Update README.md (#3) 1a758c5
2024-04-04 update tokenizer_config.json 60441b8
2024-03-29 Update README.md (#2) 36b2833
2024-03-28 Update README.md e8c04bd
2024-03-28 Update README.md 11489a5
2024-03-28 Update README.md f239983
2024-03-28 Update README.md 913247a
2024-03-28 Update README.md cfa0b3a
2024-03-28 Update config.json e533522
2024-03-28 Update README.md a9d1cf7
2024-03-28 Update README.md b59a11c
2024-03-21 Update config.json 40b8b73
2024-03-15 Update README.md ff81856
2024-03-15 Update README.md 9f55e3d
2024-03-14 fix config 3e4774b
2024-03-04 fix class name 0ac224a
2024-02-29 remove special_tokens_map.json 16f77e2
2024-02-29 upload model & configs. ce35c8d
2024-02-29 initial commit d2755d7

Articles and coverage

7 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Official2

Qwen1.5-Moe: Matching 7B Model Performance with 1/3 Activated Parameters
qwenlm.github.io2024-03-28officialauthorityHN front pageabout the Qwen1.5 familyHN 104 pts · 10 comments1,495 words
Qwen1.5-MoE-A2.7B matches 7B model performance with only 2.7 billion activated parameters using a 64-expert configuration. The model incorporates 4 shared experts alongside 60 routing experts and requires source installation via git for Hugging Face support. It achieves a 75% decrease in training expenses and accelerates inference speed by a factor of 1.74 compared to Qwen1.5-7B.written, reviewed, cited
  • The model partitions a single FFN into segments to generate 64 experts without increasing the total parameter count.
  • Training from scratch is inefficient, so the team repurposes the existing Qwen-1.8B model to initialize Qwen1.5-MoE-A2.7B.
  • The Qwen1.5-MoE-A2.7B-Chat-GPTQ-Int4 quantized variant is available, but AWQ support is currently not supported.
Qwen1.5
qwenlm.github.io2024-02-05officialauthorityabout the Qwen1.5 familyHN 5 pts · 2 comments3,793 words
Qwen1.5 is a new open-source model series released before the Chinese New Year, offering base and chat models in six sizes ranging from 0.5B to 110B parameters. The Qwen1.5-72B model outperforms Llama2-70B across all benchmarks, and the series features improved multilingual and human-alignment capabilities. All models now uniformly support a context length of up to 32768 tokens, and the Qwen1.5-72B-Chat model is recommended for initial testing.written, reviewed, cited
All coverage tracked for this model (7) — append-only, nothing is ever removed
TitleSourcePublishedFirst seenSignals
Qwen1.5-110Bqwenlm.github.io official2024-04-262026-10-07hn
Qwen1.5-110B Previewtwitter.com2024-04-202026-10-07hn
Qwen1.5-32B: Fitting the Capstone of the Qwen1.5 Language Model Seriesqwenlm.github.io official2024-04-052026-10-07hn
Qwen1.5-Moe: Matching 7B Model Performance with 1/3 Activated Parametersqwenlm.github.io official2024-03-282026-10-07hn
Run Alicloud's Qwen LLM with a 2MB cross GPU/CPU rust appsecondstate.io2024-02-282026-10-07hn
Complete tutorial: Run Qwen LLM across your local machines without Python/C++secondstate.io2024-02-222026-10-07hn
Qwen1.5qwenlm.github.io official2024-02-052026-10-07hn

Similar models

3 models

Similar models

  • DeepSeek-Coder-V2-Lite-Instruct · 15.7B
    same size band (L), also MoE, same task (text-generation)
  • DeepSeek-V2-Lite · 15.7B
    same size band (L), also MoE, same task (text-generation)
  • gpt-oss-20b · 20.9B
    same size band (L), also MoE, same task (text-generation), more providers, permissive license

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:40 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:46 UTC · Summary: 2026-10-07 15:12 UTC · Last verified 2026-10-25

Follow Qwen1.5-MoE-A2.7B

Watch this model

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.