The story so far
Weights, every dated article, and repo commits on one rail.
10 repo commits since the weights landed
Teal is the weights (2023-12-19), grey ticks are repo commits.
w2v-bert-2.0
580M parameters, text in, embeddings out.
Verified 2026-10-25 Rank #57 in Embeddings · score 87.5 method v1 Sources 3 Downloads 30d 1.1M · #104 of 619 tracked
Changed this week: 1 new quantized build on the Hub (2 in all).2026-10-24
Do people use it?1.1Mdownloads, 30 days#104 of 619 we track · top 17%+2 likes, downloads −2% this week242 likesHugging Face2026-10-24
Where it's used
Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.
No API provider serves it, so its downloads are people running it themselves: 2 quantized builds and 100 Spaces.
- As a service0no API provider listed
- On own hardware2quantized builds
- Built on544542 fine-tunes · 2 adapters · 0 merges
- In apps100+Hugging Face Spaces
Top Spaces running it: IndexTeam/IndexTTS-2-Demo ♥ 849 · amphion/maskgct ♥ 260 · embedl/hfviewer ♥ 181 · sarulab-speech/sidon_demo_beta ♥ 38
Will it fit?
Weights only, an estimate: leave room for the KV cache. Method
Fits a 16 GB card at INT4
About 0.3 GB at INT4, 0.7 GB at FP8, 1.4 GB at BF16: weights only, so leave room for the KV cache.
Estimate params × bytes per param × 1.2 · how we estimate
What people built on it
546 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.
546 derivatives on the Hub, most of them fine-tunes
From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.
Reference
Everything else, folded: the full tables, every source.
What the card says
2 cited sentences
Summary generated, cited
W2v-BERT 2.0 is a 580M parameter Conformer-based speech encoder used for feature extraction.[1][2] The model was pre-trained on 4.5M hours of unlabeled audio data and requires finetuning for tasks like Automatic Speech Recognition.[3]
- parameters human
- “Conformer-based [W2v-BERT 2.0 speech encoder](#w2v-bert-2…”
- “pre-trained on 4.5M hours of unlabeled audio data”
Generated 2026-10-07 15:03 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.
Use-case evidence
3 use cases
What it is good at 3 use cases
Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.
Facts
18 fields
Listed in Embeddings
Facts
- w2v-bert-2.0 has 580M parameters. (safetensors metadata, observed 2026-10-24)
- License: MIT (friction: none). Permissive. (license file, observed 2026-10-24)
- Created 2023-12-19, last modified 2024-01-25. (HF API, observed 2026-10-24)
| Pipeline tag | feature-extraction |
|---|---|
| Library | transformers |
| Architectures | Wav2Vec2BertModel |
| Model type | wav2vec2-bert |
| Dtype | float32 |
| Params by dtype | F32: 580,493,120 |
| Size band | S |
| MoE | no |
| Languages | af, am, ar, as, az, be, bn, bs, bg, ca, cs, zh, cy, da, de, el, en, et, fi, fr, or, om, ga, gl, gu, ha, he, hi, hr, hu, hy, ig, id, is, it, jv, ja, kn, ka, kk, mn, km, ky, ko, lo, ln, lt, lb, lg, lv, ml, mr, mk, mt, mi, my, nl, nb, ne, ny, oc, pa, ps, fa, pl, pt, ro, ru, sk, sl, sn, sd, so, es, sr, sv, sw, ta, te, tg, tl, th, tr, uk, ur, uz, vi, wo, xh, yo, ms, zu, ary, arz, yue, kea |
| Papers | arXiv:2312.05187 |
| License | MIT (mit) |
| Hub snapshot | 2026-10-25 |
Servability
vLLM · SGLang
Servability
vLLM: not in registry · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: not in registry · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed
vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.
Hardware fit
3 estimates
Hardware fit ESTIMATE
Weights-only memory for w2v-bert-2.0: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.
| BF16 (2 bytes/param) | 1.4 GB · Edge (<4 GB) |
|---|---|
| FP8 (1 byte/param) | 0.7 GB · Edge (<4 GB) |
| INT4 (0.5 bytes/param) | 0.3 GB · Edge (<4 GB) |
Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.
All variants
40 variants
Lineage
finetune: 542adapter: 2quantized: 2
Top variants (40 shown, ranked by downloads)
| Variant | Relation | Quant | Params | Band | License | Author | Likes | Downloads |
|---|---|---|---|---|---|---|---|---|
| badrex/Ethio-ASR-multilingual-600M | finetune | — | 606M | S | cc-by-4.0 | badrex | 4 | 330.3k |
| obadx/muaalem-model-v3_2 | finetune | — | 606M | S | mit | obadx | 11 | 86k |
| badrex/w2v-bert-2.0-kinyarwanda-asr | finetune | — | 581M | S | cc-by-4.0 | badrex | 6 | 69.7k |
| obadx/recitation-segmenter-v2 | finetune | — | 580M | S | mit | obadx | 4 | 25.8k |
| classla/wav2vecbert2-filledPause | finetune | — | 580M | S | apache-2.0 | classla | 1 | 24.3k |
| hon9kon9ize/wav2vec2bert-jyutping | finetune | — | 606M | S | cc-by-sa-4.0 | hon9kon9ize | 5 | 14.4k |
| badrex/Ethio-ASR-amharic | finetune | — | 606M | S | cc-by-4.0 | badrex | 2 | 14.4k |
| classla/Wav2Vec2BertPrimaryStressAudioFrameClassifier | finetune | — | 580M | S | cc-by-sa-4.0 | classla | 1 | 10.4k |
| Yehor/w2v-bert-uk-v2.1 | finetune | — | 606M | S | apache-2.0 | Yehor | 3 | 8.4k |
| sarulab-speech/sidon_raw_weight | adapter | — | — | — | mit | sarulab-speech | 3 | 6.9k |
| Yehor/w2v-bert-uk | finetune | — | 606M | S | apache-2.0 | Yehor | 8 | 4.3k |
| sazzadul/Shrutimala_Bangla_ASR | finetune | — | 606M | S | cc-by-sa-4.0 | sazzadul | 2 | 3.4k |
| GaborMadarasz/w2v-bert-2.0-hungarian-CV24.0 | finetune | — | 606M | S | mit | GaborMadarasz | 0 | 2.9k |
| badrex/Ethio-ASR-tigrinya | finetune | — | 606M | S | cc-by-4.0 | badrex | 1 | 2.6k |
| vrclc/W2V2-BERT-Malayalam | finetune | — | 606M | S | mit | vrclc | 6 | 2.2k |
| vrclc/W2V2-BERT-withLM-Malayalam | finetune | — | 606M | S | mit | vrclc | 3 | 2.2k |
| badrex/w2v-bert-2.0-malagasy-asr | finetune | — | 606M | S | cc-by-4.0 | badrex | 4 | 2k |
| Bajiyo/w2v-bert-2.0-nonstudio_and_studioRecords_final | finetune | — | 606M | S | mit | Bajiyo | 5 | 1.9k |
| Yehor/w2v-bert-uk-v2.1-fp16 | finetune | -fp16 | 606M | S | apache-2.0 | Yehor | 1 | 1.7k |
| YonaKhine/finetuned-w2v2-bert-burmese-asr | finetune | — | 606M | S | mit | YonaKhine | 2 | 1.4k |
| badrex/w2v-bert-2.0-shona-asr | finetune | — | 606M | S | cc-by-4.0 | badrex | 3 | 1.3k |
| badrex/Ethio-ASR-oromo | finetune | — | 606M | S | cc-by-4.0 | badrex | 4 | 1.3k |
| badrex/w2v-bert-2.0-zulu-asr | finetune | — | 581M | S | cc-by-4.0 | badrex | 2 | 1.4k |
| sarulab-speech/sidon-v0.1 | finetune | — | — | — | mit | sarulab-speech | 28 | 0 |
| CLEAR-Global/w2v-bert-2.0-yoruba_naijavoices_500h | finetune | — | 606M | S | mit | CLEAR-Global | 1 | 1.2k |
| cdactvm/w2v-bert-2.0-hindi_new | finetune | — | 606M | S | mit | cdactvm | 0 | 1.3k |
| trick4kid/w2v-bert-2.0-vietnamese-CV16.0 | finetune | — | 606M | S | mit | trick4kid | 0 | 1.2k |
| whitefox123/w2v-bert-2.0-arabic-4 | finetune | — | 606M | S | mit | whitefox123 | 0 | 1.1k |
| ErikMkrtchyan/w2v-bert-2.0-armenian-CV20.0 | finetune | — | 606M | S | mit | ErikMkrtchyan | 0 | 1k |
| CLEAR-Global/w2v-bert-2.0-igbo_naijavoices_500h | finetune | — | 606M | S | mit | CLEAR-Global | 1 | 928 |
| ayymen/w2v-bert-2.0-waxal_lin_sna-1epoch | finetune | — | 606M | S | mit | ayymen | 0 | 963 |
| CLEAR-Global/w2v-bert-2.0-hausa_579_993h_yourtts | finetune | — | 606M | S | mit | CLEAR-Global | 1 | 880 |
| ihanif/pashto-asr-v3 | finetune | — | 606M | S | mit | ihanif | 1 | 815 |
| douyeszn/w2vbert-lin-waxal-aug-ft | finetune | — | 606M | S | apache-2.0 | douyeszn | 1 | 809 |
| mikr/w2v-bert-2.0-czech-colab-cv16 | finetune | — | 606M | S | mit | mikr | 2 | 743 |
| yuriyvnv/WAVe-1B-Multimodal-PT | finetune | — | 879M | S | apache-2.0 | yuriyvnv | 8 | 436 |
| Xenova/w2v-bert-2.0 | quantized | — | — | — | — | Xenova | 2 | 718 |
| badrex/w2v-bert-2.0-swahili-asr | finetune | — | 581M | S | cc-by-4.0 | badrex | 0 | 809 |
| badrex/w2v-bert-2.0-kikuyu-asr | finetune | — | 581M | S | cc-by-4.0 | badrex | 0 | 805 |
| asr-malayalam/malayalam_combined_ | finetune | — | 606M | S | mit | asr-malayalam | 0 | 768 |
Repository changelog
10 commits
Repository changelog 10 commits
| Date | Commit | SHA |
|---|---|---|
| 2024-01-25 | Update README.md (#14) | da985ba |
| 2024-01-16 | Upload model (#12) | 6b1c0b3 |
| 2024-01-16 | Upload feature extractor (#13) | 3dadbe0 |
| 2024-01-04 | Upload Wav2Vec2BERTForPreTraining (#8) | 8353d27 |
| 2024-01-04 | Upload feature extractor (#9) | e531313 |
| 2023-12-19 | Update README.md | 15f40b8 |
| 2023-12-19 | Update README.md (#2) | 0852d0f |
| 2023-12-19 | Update README.md (#1) | 8de6050 |
| 2023-12-19 | Initial commit. | 41b77cb |
| 2023-12-19 | initial commit | 54f8052 |
Articles and coverage
2 pieces
Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.
Analysis2
arxiv.orgauthority4,535 words
A speaker verification system utilizes w2v-BERT 2.0, a 600 million parameter model trained on 4.5 million hours of data across 143 languages, achieving 0.12% and 0.55% EER on Vox1-O and Vox1-H test sets respectively. The architecture employs a Multi-scale Feature Aggregation structure with Layer Adapters and LoRA, while knowledge distillation guided structured pruning reduces model size by 80% with only 0.04% EER degradation. Pruning optimization uses the Hard Concrete distribution to handle the discrete L0 regularization term during sparsity control.written, reviewed, cited
- The w2v-BERT 2.0 encoder consists of 24 conformer layers integrated with both contrastive learning and masked language modeling.
- LoRA adaptation is applied specifically to the query and value weights of the PTM's self-attention module.
- Pruning parameters are modeled as random variables governed by the Hard Concrete distribution to enable differentiable L0 regularization.
emergentmind.com2,125 words
w2v-BERT 2.0 is a 24-layer Conformer backbone trained with contrastive learning and masked prediction objectives across 143 languages. Downstream reports cite training data scales ranging from 4.5 million to 450 million hours of unlabeled speech. The model is designed to be used as a multi-scale representational hierarchy where intermediate layers are reweighted or aggregated for downstream tasks.written, reviewed, cited
- The Whale ASR system integrates w2v-BERT 2.0 as a front-end within a 1.87B-parameter encoder-decoder architecture.
- Empirical analysis identifies layers 15–19, with a peak at layer 18, as especially informative for mild cognitive impairment classification.
- Structured pruning reduces the model size by 80% while causing only 0.04% EER degradation on Vox1-O and Vox1-H test sets.
Papers
1 paper
Papers 1
arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).
- Seamless Communication, Loïc Barrault, Yu-An Chung +62 · 2023-12-08 · PDF · HF paper page (14 upvotes)
Similar models
3 models
Similar models
- multilingual-e5-large · 560M same size band (S), same task (feature-extraction), more providers, permissive license
- Qwen3-Embedding-0.6B · 596M same size band (S), same task (feature-extraction), more providers, permissive license
- multilingual-e5-large-instruct · 560M same size band (S), same task (feature-extraction), more providers, permissive license
Sources
3 sources
Sources
Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:35 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:03 UTC · Last verified 2026-10-25
Follow w2v-bert-2.0
Watch this model
- /feeds/models/facebook--w2v-bert-2.0.xml(RSS 2.0) · every change to w2v-bert-2.0
- /feeds/families/facebook--w2v-bert-2.0.xml(RSS 2.0) · every change across the w2v-bert-2.0 family
Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.