Models › opt family

facebook Other out 4 years ago

Text in, text out, 2K tokens of context.

Verified 2026-10-25 Rank #133 in LLMs · score 94.3 method v1 Sources 3 Downloads 30d 5.4M · #28 of 619 tracked

Changed this week: 1 new quantized build on the Hub (14 in all).2026-10-24

Can I use it?OtherCustom license; read the repo LICENSE before commercial hostinglicence file2026-10-24
Do people use it?5.4Mdownloads, 30 days#28 of 619 we track · top 5%+3 likes, downloads −7% this week332 likesHugging Face2026-10-24

Worth reading

Picked by rule from 7 tracked pieces: the lab's own post, the most shared, a field report, the latest.

7 best of 7 tracked

The 2 picks above, plus these 5. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

  1. ArticleOpen Pre-Trained Transformer (OPT)emergentmind.com · undated
  2. ArticleArchitecture graph for facebook/opt-125mhfviewer.com · undated
  3. ArticleOPT-125M by facebook | The GTM Directorythegtmdirectory.com · undated
  4. ArticleInstruction Tuning OPT-125Mdebuggercafe.com · undated
  5. ArticleOPT-125M by Meta AI : A Complete Guide to Open Pre-Trained Transformer Models - Vanita.aivanitaai.com · 2026-02-15

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 14 quantized builds and 100 Spaces.

  • As a service0no API provider listed
  • On own hardware14quantized builds
  • Built on421100 fine-tunes · 321 adapters · 0 merges
  • In apps100+Hugging Face Spaces

Top Spaces running it: badayvedat/LLaVA ♥ 425 · LanguageBind/Video-LLaVA ♥ 234 · embedl/hfviewer ♥ 181 · THUdyh/Oryx ♥ 102

The story so far

Weights, every dated article, and repo commits on one rail.

2 dated articles and 30 repo commits so far

Teal is the weights (2022-05-11), amber triangles are 2 dated articles (each links to it), grey ticks are repo commits.

JunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2023-09-15: 1 repo commit2023-01-24: 1 repo commit2022-06-22: 1 repo commit2022-06-21: 1 repo commit2022-06-16: 1 repo commit2022-06-03: 4 repo commits2022-05-26: 1 repo commit2022-05-25: 2 repo commits2022-05-24: 1 repo commit2022-05-19: 1 repo commit2022-05-18: 2 repo commits2022-05-17: 1 repo commit2022-05-16: 3 repo commits2022-05-13: 2 repo commits2022-05-12: 8 repo commitsGPT-2 vs OPT-125M — same skeleton, completely different internal dynamics - Models - Hugging Face Forums · 2026-05-29OPT-125M by Meta AI : A Complete Guide to Open Pre-Trained Transformer Models - Vanita.ai · 2026-02-15Weights 2022-05-11 · 2022-05-11Weights 2022-05-11now
JunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2023-09-15: 1 repo commit2023-01-24: 1 repo commit2022-06-22: 1 repo commit2022-06-21: 1 repo commit2022-06-16: 1 repo commit2022-06-03: 4 repo commits2022-05-26: 1 repo commit2022-05-25: 2 repo commits2022-05-24: 1 repo commit2022-05-19: 1 repo commit2022-05-18: 2 repo commits2022-05-17: 1 repo commit2022-05-16: 3 repo commits2022-05-13: 2 repo commits2022-05-12: 8 repo commitsGPT-2 vs OPT-125M — same skeleton, completely different internal dynamics - Models - Hugging Face Forums · 2026-05-29OPT-125M by Meta AI : A Complete Guide to Open Pre-Trained Transformer Models - Vanita.ai · 2026-02-15Weights 2022-05-11 · 2022-05-11Weights 2022-05-11now

What people built on it

435 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

435 derivatives on the Hub, most of them adapters

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

14 quantized (Hugging Face model tree)100 fine-tunes (Hugging Face model tree)321 adapters (Hugging Face model tree)14 quantized100 fine-tunes321 adapters
14 quantized (Hugging Face model tree)100 fine-tunes (Hugging Face model tree)321 adapters (Hugging Face model tree)14 quantized100 fine-tunes321 adapters

All variants →

Reference

Everything else, folded: the full tables, every source.

What the card says

1 cited sentence

Summary generated, cited

OPT-125m is a decoder-only pre-trained transformer model designed for text generation.[1]

  1. “decoder-only pre-trained transformers”

Generated 2026-10-07 15:02 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

1 use case

What it is good at 1 use case

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

18 fields

Listed in LLMs

Facts

Pipeline tagtext-generation
Librarytransformers
ArchitecturesOPTForCausalLM
Model typeopt
Dtypefloat16
MoEno
Context length2,048 (config.json max_position_embeddings)
Languagesen
PapersarXiv:2205.01068, arXiv:2005.14165
LicenseOther (other)
Hub snapshot2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · OPTForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · OPTForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

All variants

40 variants

Lineage

Original model Family: facebook/opt 435 derivatives on the Hub

adapter: 321finetune: 100quantized: 14

Top variants (40 shown, ranked by downloads)

Relation
Quant
40 rows
Variant Relation Quant Params Band License Author Likes Downloads
peft-internal-testing/opt-125m-dummy-lora adapter — — — — peft-internal-testing 0 139.2k
pszemraj/opt-125m-email-generation finetune — 125M S other pszemraj 3 9.7k
jwongTensora/test-repo finetune — 125M S other jwongTensora 0 2k
Xenova/opt-125m quantized — — — — Xenova 0 1.4k
nicholasKluge/Aira-OPT-125M finetune — 125M S other nicholasKluge 0 1.2k
ryota-komatsu/SylReg-LM-85M finetune — 132M S — ryota-komatsu 0 1k
Cheng98/opt-125m-sst2 finetune — — — other Cheng98 0 934
mendeza/opt-125m-synthetic-finetuned finetune — 125M S other mendeza 0 769
Herocat/opt-125m-finetuned-wikitext2 finetune — 125M S other Herocat 0 747
SystemAdmin123/opt-125m finetune — 125M S other SystemAdmin123 0 737
SirSahOl/opt-125m-chat-mlx-16bit finetune mlx 164M S other SirSahOl 0 641
ansilmbabl/ft-opt-125m-customer-chatbot-4-zephyr-tokenizer finetune — 125M S other ansilmbabl 1 556
krushnakant27/bertQA finetune — 125M S other krushnakant27 0 570
jeanlucmarsh/opt-125m-pattern-based_finetuning_with_lora-mnli-mm-d3_fs2 adapter — — — other jeanlucmarsh 0 536
Cheng98/opt-125m-qnli finetune — — — other Cheng98 0 513
ybelkada/test-axolotl-onecyclelrschedulertrainer finetune — 125M S other ybelkada 0 495
Cheng98/opt-125m-rte finetune — — — other Cheng98 0 491
ybelkada/test-axolotl-axolotlmambatrainer finetune — 125M S other ybelkada 0 488
ybelkada/test-axolotl finetune — 125M S other ybelkada 0 462
ellen625/opt125_wiki_rlo_k50 finetune — 125M S other ellen625 0 460
Cheng98/opt-125m-qqp finetune — — — other Cheng98 0 459
ellen625/opt125_wiki_ce finetune — 125M S other ellen625 0 451
kashif/gkd_openassistant-guanaco finetune — 125M S other kashif 0 445
tomaszki/mail_fixer finetune — — — other tomaszki 0 431
SebastianSchramm/opt-125m-sft-full finetune — 125M S other SebastianSchramm 0 426
ellen625/opt125_wiki_rlo_k3 finetune — 125M S other ellen625 0 426
ellen625/opt_125_wiki_k10 finetune — 125M S other ellen625 0 421
monsterapi/opt125M_alpaca adapter — — — — monsterapi 0 416
ellen625/ft_opt125_ce finetune — 125M S other ellen625 0 412
gnumanth/tmp_trainer finetune — 125M S other gnumanth 0 411
ansilmbabl/ft-opt-125m-customer-chatbot-3 finetune — 125M S other ansilmbabl 0 405
ansilmbabl/ft-opt-125m-customer-chatbot-2 finetune — 125M S other ansilmbabl 0 381
error577/5f0159c4-1008-4527-9092-4ee6e6b9e663 adapter — — — other error577 0 333
VisitationAI/opt125-llama-visitation finetune — 125M S llama3 VisitationAI 0 309
AlexKoff88/opt-125m-openvino-4bit finetune -4bit — — other AlexKoff88 0 285
quansuv/opt-125m-finetuned-wikitext-103 finetune — 125M S other quansuv 0 262
elliottfitzgerald/opt-125m-finetuned-mnli-mm adapter — 125M S other elliottfitzgerald 0 217
SirSahOl/opt-125m-chat-mlx-4bit quantized mlx 164M S other SirSahOl 0 216
mahojo/opt-125m-cluster-v2 finetune — 125M S other mahojo 0 171
Albatu1773/opt-125m-fine-tuned-lora-model adapter — — — — Albatu1773 1 112

Repository changelog

30 commits
changed in the last 7 daysOther: 2026-10-24all changes

Repository changelog 30 commits

DateCommitSHA
2023-09-15 Update README.md (#38) 27dcfa7
2023-01-24 Adding generation config file(s) 3d2b5f2
2022-06-22 Update README.md (#5) 934b6a0
2022-06-21 correct checkpoints see: https://github.com/facebookresearch/metaseq/pull/164 507a399
2022-06-16 Update TF weights (#2) 38f3ba0
2022-06-03 Add flax_weights 80cabbb
2022-06-03 Add tf_weights c681a93
2022-06-03 Update README.md 449ab1c
2022-06-03 add model 6fb0883
2022-05-26 Change BOS token from 0 to 2 as BOS token is equal to EOS for OPT. See: https://github.com/huggingface/transformers/issues/17431 (#1) da93426
2022-05-25 Update tokenizer_config.json 1c370c8
2022-05-25 Upload tf_model.h5 46941ec
2022-05-24 Update tf weights 75f3881
2022-05-19 Update tf weights 71aed4d
2022-05-18 Upload tf_model.h5 13ec9fc
2022-05-18 Update config.json c1c6d4a
2022-05-17 add model fae5403
2022-05-16 Set commercial attribute to False 403005f
2022-05-16 Update license 5c42793
2022-05-16 Update licence 83a28f5
2022-05-13 Temporarily deactivate the inference API 2affb45
2022-05-13 Update config.json 74b53f9
2022-05-12 Update README.md 0770005
2022-05-12 Update README.md 853beb2
2022-05-12 Create README.md 97c3f46
2022-05-12 finish 04e12f0
2022-05-12 finish fde505d
2022-05-12 Merge branch 'main' of https://huggingface.co/facebook/opt-125m into main 141c875
2022-05-12 add tokenizer c29996b
2022-05-12 Update config.json 3bbf6e4

Articles and coverage

7 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Analysis6

emergentmind.comabout the opt family1,129 words
Meta AI released the Open Pre-Trained Transformer (OPT), a suite of decoder-only models ranging from 125M to 175B parameters pre-trained on 180 billion English tokens. The 175B variant generated approximately 75 tons CO₂-equivalent emissions, roughly one-seventh of GPT-3's footprint, while smaller models utilize an Apache 2.0 license. Researchers can access the full training pipeline via the 'metaseq' codebase and a comprehensive infrastructure logbook.written, reviewed, cited
  • Training the 175B model required approximately two months, including 35 manual and about 70 automatic restarts due to hardware failures.
  • OPT-175B is licensed for non-commercial research, whereas smaller models use Apache 2.0.
  • All models accept up to 2048 tokens as context.
hfviewer.com4,498 words
hfviewer renders an interactive architecture graph for the Hugging Face model facebook/opt-125m, visualizing its real structure with nodes for embeddings, attention blocks, and feed-forward layers. The model uses 12 transformer layers with a hidden size of 768 and 12 attention heads, while feed-forward layers employ an intermediate size of 3,072. It features a vocabulary of 50,272 tokens and a context window of up to 2,048 tokens.written, reviewed, cited
  • The graph is trace-backed where executable or derived from reviewed configuration and source code otherwise.
  • Engineers can replace huggingface.co with hfviewer.com in the url to view the model architecture graph.
  • The hfviewer system discovers the architecture graph automatically when it appears in the model card README.md.
llm-explorer.com
thegtmdirectory.com990 words
Meta released OPT-125M on 2022-05-11 as a 125M parameter open-weight decoder-only model with fully open weights paired with training code and a detailed infrastructure logbook. Builders fine-tune it on modest hardware without a GPU cluster, though version pinning is required to avoid compatibility regressions with Hugging Face Transformers. The model yields unreliable outputs for production inference or tasks requiring strong zero-shot performance without significant fine-tuning.written, reviewed, cited
  • The model is part of a decoder-only suite spanning 125M to 175B parameters, allowing researchers to scale experiments within the same family.
  • Weights are available under CC BY-NC 4.0 for self-hosting in research and non-commercial settings, limiting commercial deployment.
  • Generation quality is poor out of the box, especially with sampling enabled, making it unsuitable for immediate production use.
debuggercafe.com3,169 words
vanitaai.com2026-02-15940 words

Community1

discuss.huggingface.co2026-05-29230 words
GPT-2 and OPT-125M share a skeleton of 12 layers and 768 dimensions but exhibit radically different internal dynamics during inference. GPT-2 commits early around layer 8 of 12, making it better for tasks needing decisive, confident output like classification or extraction. OPT-125M maintains uncertainty much longer with a low top1 of ~0.03, suiting exploration, creativity, or safety margin needs, and remains more stable under perturbation during fine-tuning.written, reviewed, cited

Papers

2 papers

Papers 2

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Similar models

3 models

Similar models

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:35 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:02 UTC · Last verified 2026-10-25

Follow opt-125m

Watch this model

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.