Model Follower.

Up to date

Rebuilt
2026-10-25 06:54 UTC
Release radar
117 on watch
Hub snapshot
2026-10-25
Re-checked
daily · every number sourced and dated

Models › gpt-neo family

gpt-neo-125m

Hugging Face ↗ Follow changes

EleutherAI MIT out 5 years ago

150M parameters, text in, text out, 2K tokens of context.

Verified 2026-10-25 Rank #347 in LLMs · score 81.4 method v1 Sources 3 Downloads 30d 467.2k · #168 of 619 tracked

Can I run it?0.1GB at INT4Estimatefits a 16 GB cardmethod
Can I use it?MITPermissivelicence file2026-10-24
Do people use it?467.2kdownloads, 30 days#168 of 619 we track · top 27%+0 likes, downloads −5% this week231 likesHugging Face2026-10-24

Worth reading

Picked by rule from 15 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Most discussed · 561 HN pts

GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models

The GPT-Neo repository offers an archival implementation of model parallel GPT-3-style models using the mesh-tensorflow library, superseded by GPT-NeoX for GPU efficiency.

  • Training and inference is officially supported on TPU and should work on GPU as well.
  • Google colab provides tpu-v8s for free, which should be enough to finetune our models up to GPT3XL (1.5B parameter) sizes.
github.com 2021-03-21 561 HN pts · 127 comments 3k words about the gpt neo family family

15 best of 15 tracked

The 2 picks above, plus these 13. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

  1. ArticleGPT-Neo – Building a GPT-3-sized model, open source and freeeleuther.ai · 2021-01-18 · 725 HN pts · official
  2. ArticleFew-Shot Learning in Practice: GPT-Neo & 'HuggingFace' Accelerated Inference APIhuggingface.co · 2021-06-04 · 69 HN pts
  3. ArticleUsing GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learningnlpcloud.io · 2021-11-10 · 24 HN pts
  4. DiscussionH3 – Outperforming GPT-Neo-2.7B with only 2 attention layerstwitter.com · 2023-01-23
  5. ArticleGPT-Neo vs. GPT-3: Are Commercialized NLP Models That Much Better?medium.com · 2021-04-09 · 2 HN pts
  6. ArticleGPT-Neo on a M1 (macOS 12)lazycoder.ro · 2022-02-03 · 5 HN pts
  7. ArticleGPT-3's free alternative GPT-Neo is something to be excited aboutventurebeat.com · 2021-05-15 · 5 HN pts
  8. ArticleGPT-3 open-source alternatives: GPT-Neo and GPT-Jnlpcloud.io · 2021-12-14 · 4 HN pts
  9. ArticleNovelAI, a new AI storyteller being built on GPT-Neonovelai.medium.com · 2021-05-31 · 2 HN pts
  10. ArticlePretrain GPT-Neo for Open Source GitHub Copilot Modeldiscuss.huggingface.co · 2021-07-03
  11. ArticleHow to monitor the performance of a fine-tuned GPT-Neo Model in Productionblog.cerebrium.ai · 2023-02-28
  12. DiscussionFew-shot learning NLP with GPT-J and GPT-Neonews.ycombinator.com · 2021-07-22
  13. DiscussionHow to run inference with GPT-Neo for freetwitter.com · 2021-04-05

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 4 quantized builds and 100 Spaces.

  • As a service0no API provider listed
  • On own hardware4quantized builds
  • Built on282183 fine-tunes · 99 adapters · 0 merges
  • In apps100+Hugging Face Spaces

Top Spaces running it: Sharathhebbar24/One-stop-for-Open-source-models ♥ 96 · KBaba7/Quant ♥ 51 · yuntian-deng/latex2im ♥ 26 · FallnAI/Quantize-HF-Models ♥ 19

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 16 GB card at INT4

About 0.1 GB at INT4, 0.2 GB at FP8, 0.4 GB at BF16: weights only, so leave room for the KV cache.

16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 0.1 GB of weights (estimate; KV cache and activations extra)INT40.1 GBFP8: about 0.2 GB of weights (estimate; KV cache and activations extra)FP80.2 GBBF16: about 0.4 GB of weights (estimate; KV cache and activations extra)BF160.4 GB
16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 0.1 GB of weights (estimate; KV cache and activations extra)INT40.1 GBFP8: about 0.2 GB of weights (estimate; KV cache and activations extra)FP80.2 GBBF16: about 0.4 GB of weights (estimate; KV cache and activations extra)BF160.4 GB

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

Most coverage came in the first two weeks

Teal is the weights (2022-03-02), amber triangles are 10 dated articles (each links to it), grey ticks are repo commits.

FebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2024-01-31: 2 repo commits2023-07-09: 1 repo commit2023-06-21: 1 repo commit2022-12-08: 1 repo commit2022-08-24: 1 repo commit2021-12-31: 2 repo commits2021-09-11: 3 repo commits2021-07-04: 1 repo commit2021-05-20: 2 repo commits2021-05-18: 1 repo commit2021-04-17: 1 repo commit2021-04-06: 3 repo commitsGPT-Neo – Building a GPT-3-sized model, open source and free · 2021-01-18GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models · 2021-03-21Few-Shot Learning in Practice: GPT-Neo & 'HuggingFace' Accelerated Inference API · 2021-06-04Using GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learning · 2021-11-10Hugging Face Transformers – Big Model Inference, Bloom, GPT Neo-X, LongT5 etc. · 2022-06-16GPT-Neo vs. GPT-3: Are Commercialized NLP Models That Much Better? · 2021-04-09GPT-3's free alternative GPT-Neo is something to be excited about · 2021-05-15GPT-Neo on a M1 (macOS 12) · 2022-02-03GPT-3 open-source alternatives: GPT-Neo and GPT-J · 2021-12-14NovelAI, a new AI storyteller being built on GPT-Neo · 2021-05-31Weights 2022-03-02 · 2022-03-02Weights 2022-03-02now
FebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2024-01-31: 2 repo commits2023-07-09: 1 repo commit2023-06-21: 1 repo commit2022-12-08: 1 repo commit2022-08-24: 1 repo commit2021-12-31: 2 repo commits2021-09-11: 3 repo commits2021-07-04: 1 repo commit2021-05-20: 2 repo commits2021-05-18: 1 repo commit2021-04-17: 1 repo commit2021-04-06: 3 repo commitsGPT-Neo – Building a GPT-3-sized model, open source and free · 2021-01-18GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models · 2021-03-21Few-Shot Learning in Practice: GPT-Neo & 'HuggingFace' Accelerated Inference API · 2021-06-04Using GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learning · 2021-11-10Hugging Face Transformers – Big Model Inference, Bloom, GPT Neo-X, LongT5 etc. · 2022-06-16GPT-Neo vs. GPT-3: Are Commercialized NLP Models That Much Better? · 2021-04-09GPT-3's free alternative GPT-Neo is something to be excited about · 2021-05-15GPT-Neo on a M1 (macOS 12) · 2022-02-03GPT-3 open-source alternatives: GPT-Neo and GPT-J · 2021-12-14NovelAI, a new AI storyteller being built on GPT-Neo · 2021-05-31Weights 2022-03-02 · 2022-03-02Weights 2022-03-02now

What people built on it

286 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

286 derivatives on the Hub, most of them fine-tunes

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

4 quantized (Hugging Face model tree)183 fine-tunes (Hugging Face model tree)99 adapters (Hugging Face model tree)4 quantized183 fine-tunes99 adapters
4 quantized (Hugging Face model tree)183 fine-tunes (Hugging Face model tree)99 adapters (Hugging Face model tree)4 quantized183 fine-tunes99 adapters

All variants →

Reference

Everything else, folded: the full tables, every source.

Benchmarks

8 rows

Benchmarks 8 rows

measured third-party leaderboard rowauthor-reported number from the model card or eval files (8)No confidence intervals unless the source gave them; settings differ between rows, so compare within a benchmark, not across. Date = the benchmark's own date when it gives one, else when we read it.
BenchmarkValueSetting / notesReported byDate
author-reported ARC (25-shot)22.95—model author2026-10-07
author-reported Avg.25.79—model author2026-10-07
author-reported DROP (3-shot)3.69—model author2026-10-07
author-reported GSM8K · Reason through hard problems0.3—model author2026-10-07
author-reported HellaSwag · Chat and follow instructions30.26—model author2026-10-07
author-reported MMLU · Reason through hard problems25.97—model author2026-10-07
author-reported TruthfulQA · Chat and follow instructions45.58—model author2026-10-07
author-reported Winogrande · Chat and follow instructions51.78—model author2026-10-07

What the card says

2 cited sentences

Summary generated, cited

GPT-Neo 125M is a 150M parameter transformer model designed for text generation.[1][2] It was trained on the Pile dataset as a masked autoregressive language model.[3]

  1. parameters human
  2. “GPT-Neo 125M is a transformer model”
  3. “trained on the Pile, a large scale curated dataset”

Generated 2026-10-07 15:08 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

3 use cases

What it is good at 3 use cases

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

20 fields

Listed in LLMs

Facts

Pipeline tagtext-generation
Librarytransformers
ArchitecturesGPTNeoForCausalLM
Model typegpt_neo
Params by dtypeF32: 125,198,592 · U8: 25,165,824
Size bandS
MoEno
Context length2,048 (config.json max_position_embeddings)
Languagesen
PapersarXiv:2101.00027
LicenseMIT (mit)
Hub snapshot2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: not in registry · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: not in registry · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for gpt-neo-125m: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)0.4 GB · Edge (<4 GB)
FP8 (1 byte/param)0.2 GB · Edge (<4 GB)
INT4 (0.5 bytes/param)0.1 GB · Edge (<4 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.

All variants

40 variants

Lineage

Original model Family: EleutherAI/gpt-neo 286 derivatives on the Hub

quantized: 4finetune: 183adapter: 99

Top variants (40 shown, ranked by downloads)

Relation
40 rows
Variant Relation Quant Params Band License Author Likes Downloads
Xenova/gpt-neo-125M quantized — — — — Xenova 0 2.6k
ColleenMacklin/gpt-neo-125M-couples_therapist_full finetune — — — mit ColleenMacklin 0 2.3k
Triangles/gpt-neo-125m-primordial_earth_full finetune — — — mit Triangles 0 2.1k
pszemraj/gpt-neo-125M-magicprompt-SD finetune — 176M S apache-2.0 pszemraj 1 1.2k
bhaskartripathi/GPT_Neo_Market_Analysis adapter — — — mit bhaskartripathi 3 826
ailments/gpt-neo-125M-couples_therapist_full_renamed finetune — 125M S mit ailments 0 666
bogosla/gpt-125M-CR finetune — 125M S mit bogosla 0 588
gwendolyn122/my_awesome_glue_gpt-model finetune — 125M S mit gwendolyn122 0 526
liodon-ai/gpt-neo-125m-ONNX quantized — — — other liodon-ai 0 478
KimByeongSu/gpt-neo-1.3B-cs-finetuning finetune — 125M S mit KimByeongSu 0 476
Samurai719214/gptneo-mythology-storyteller finetune — 125M S mit Samurai719214 1 398
amirali1985/gpt-neo-125m_hh_reward finetune — 125M S mit amirali1985 0 447
SanyaAhmed/llm-personality-model finetune — 125M S mit SanyaAhmed 1 374
pitangent-ds/gpt-neo-125M-tutor finetune — 125M S mit pitangent-ds 0 401
KimByeongSu/gpt-neo-125m-cs-finetuning-70000-1 finetune — 125M S mit KimByeongSu 0 387
potatomode/jokes_model2 finetune — — — mit potatomode 0 386
ethannhzhouu/EthanHorror4 finetune — — — mit ethannhzhouu 0 384
rayhanozzy/text_generation finetune — — — mit rayhanozzy 0 382
KimByeongSu/gpt-neo-125m_LAMA_TREx_finetuning_aeda finetune — — — mit KimByeongSu 0 380
kaiheilauser/GPT_Neo_llmcs_clean finetune — 125M S mit kaiheilauser 0 376
ICT3214-Group5/MD5_gpt_neo_v3.2.1 finetune — 125M S mit ICT3214-Group5 0 366
ICT3214-Group5/Cryptography_GPT_NEO_v1.0.0 finetune — 125M S mit ICT3214-Group5 1 303
KimByeongSu/gpt-neo-125m-cs-finetuning-35000-2 finetune — 125M S mit KimByeongSu 0 344
KimByeongSu/gpt-neo-125m-cs-finetuning-100000-2 finetune — 125M S mit KimByeongSu 0 343
KimByeongSu/gpt-neo-125m-lama-finetuning finetune — 125M S mit KimByeongSu 0 330
asthaaa300/results finetune — 125M S mit asthaaa300 1 280
ICT3214-Group5/temp_gpt_neo finetune — 125M S mit ICT3214-Group5 0 327
HanliangXu/my_awesome_eli5_clm-model finetune — 125M S mit HanliangXu 0 324
KimByeongSu/gpt-neo-125m-cs-finetuning-50000-3 finetune — 125M S mit KimByeongSu 0 313
Triangles/gpt-neo-125m-finetuned-philosopher_rave_10 finetune — 125M S mit Triangles 0 313
KimByeongSu/gpt-neo-125m-cs-finetuning-80000-1 finetune — 125M S mit KimByeongSu 0 310
KimByeongSu/gpt-neo-125m-cs-finetuning-25000 finetune — 125M S mit KimByeongSu 0 309
KimByeongSu/gpt-neo-125m-cs-finetuning finetune — 125M S mit KimByeongSu 0 309
KimByeongSu/gpt-neo-125m-cs-finetuning-3000-3 finetune — 125M S mit KimByeongSu 0 307
KimByeongSu/gpt-neo-125m-cs-finetuning-15000-2 finetune — 125M S mit KimByeongSu 0 304
ICT3214-Group5/MD5_gpt_neo_v1.1 finetune — 125M S mit ICT3214-Group5 0 293
Triangles/gpt-neo-125m-finetuned-philosopher_rave_100 finetune — 125M S mit Triangles 0 282
mia-llm/gpt-neo-125m-AGnews-roya finetune — 125M S — mia-llm 0 281
mia-llm/gpt-neo-125m-xsum-roya finetune — 125M S — mia-llm 0 253
serene89104/gpt-neo-125m-finetuned-cybersecurity finetune — 125M S mit serene89104 0 252

Repository changelog

19 commits

Repository changelog 19 commits

DateCommitSHA
2024-01-31 Add missing files and fix tokenizer_config.json (#6) 21def01
2024-01-31 Adding Evaluation Results (#10) 4445ae4
2023-07-09 Adding `safetensors` variant of this model (#9) 6cb0d32
2023-06-21 Update README.md c156f2b
2022-12-08 Correct metadata b983397
2022-08-24 Minor fix to example in the readme (#1) 324e21b
2021-12-31 Update README.md b559e35
2021-12-31 Updated tags to correctly link with the Pile 15f9bc1
2021-09-11 Update README.md bf56789
2021-09-11 Update README.md 4feed19
2021-09-11 Update README.md 2e2dbe5
2021-07-04 add flax model 0439444
2021-05-20 Update README.md 5e6c180
2021-05-20 Update README.md 8741c10
2021-05-18 Create README.md a420051
2021-04-17 Addition of Rust model f856adc
2021-04-06 add tokenizer files b837f54
2021-04-06 Add model c0af926
2021-04-06 initial commit 1c8841b

Articles and coverage

15 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Official2

GPT-Neo – Building a GPT-3-sized model, open source and free
eleuther.ai2021-01-18officialHN front pageabout the gpt neo familyHN 725 pts · 252 comments
Few-Shot Learning in Practice: GPT-Neo & 'HuggingFace' Accelerated Inference API
huggingface.co2021-06-04authorityabout the gpt neo familyHN 69 pts · 3 comments1,201 words
GPT-Neo enables few-shot learning for NLP tasks by requiring only 3-4 examples to achieve good results, as it is about 60x smaller than GPT-3. The 🤗 Accelerated Inference API hosts this 2.7B parameter model and offers up to 100x speedup compared to standard Transformers deployments. Engineers can control generation randomness via temperature and stop sequences using simple API calls.written, reviewed, cited
  • GPT-Neo checkpoints are trained exclusively on the Pile dataset, so performance degrades on text distributions different from its training corpus.
  • The Accelerated Inference API supports inference on over 10,000 publicly available models or private models via simple API calls.
  • Regulations like GDPR require that users receive an explanation for automatic decisions made about them by the system.

Analysis8

GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models
github.com2021-03-21authorityHN front pageabout the gpt neo familyHN 561 pts · 127 comments2,997 words
The GPT-Neo repository offers an archival implementation of model parallel GPT-3-style models using the mesh-tensorflow library, superseded by GPT-NeoX for GPU efficiency. As of August 2021, the code is no longer maintained. Two pretrained models trained on The Pile are available in 1.3B and 2.7B sizes.written, reviewed, cited
  • Training and inference is officially supported on TPU and should work on GPU as well.
  • Google colab provides tpu-v8s for free, which should be enough to finetune our models up to GPT3XL (1.5B parameter) sizes.
  • Some users have reported having difficulty getting MTF to recognize their GPUs.
Using GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learning
nlpcloud.io2021-11-10about the gpt neo familyHN 24 pts · 1 comments
Hugging Face Transformers – Big Model Inference, Bloom, GPT Neo-X, LongT5 etc.
github.com2022-06-16authorityabout the gpt neo familyHN 2 pts · 0 comments3,146 words
GPT-Neo vs. GPT-3: Are Commercialized NLP Models That Much Better?
medium.com2021-04-09about the gpt neo familyHN 2 pts · 2 comments
GPT-3's free alternative GPT-Neo is something to be excited about
venturebeat.com2021-05-15about the gpt neo familyHN 5 pts · 0 comments
GPT-Neo on a M1 (macOS 12)
lazycoder.ro2022-02-03about the gpt neo familyHN 5 pts · 0 comments557 words
GPT-3 open-source alternatives: GPT-Neo and GPT-J
nlpcloud.io2021-12-14about the gpt neo familyHN 4 pts · 0 comments
NovelAI, a new AI storyteller being built on GPT-Neo
novelai.medium.com2021-05-31about the gpt neo familyHN 2 pts · 1 comments
All coverage tracked for this model (15) — append-only, nothing is ever removed
TitleSourcePublishedFirst seenSignals
How to monitor the performance of a fine-tuned GPT-Neo Model in Productionblog.cerebrium.ai2023-02-282026-10-07hn
H3 – Outperforming GPT-Neo-2.7B with only 2 attention layerstwitter.com2023-01-232026-10-07hn
Hugging Face Transformers – Big Model Inference, Bloom, GPT Neo-X, LongT5 etc.github.com2022-06-162026-10-07hn
GPT-Neo on a M1 (macOS 12)lazycoder.ro2022-02-032026-10-07hn
GPT-3 open-source alternatives: GPT-Neo and GPT-Jnlpcloud.io2021-12-142026-10-07hn
Using GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learningnlpcloud.io2021-11-102026-10-07hn
Few-shot learning NLP with GPT-J and GPT-Neonews.ycombinator.com2021-07-222026-10-07hn
Pretrain GPT-Neo for Open Source GitHub Copilot Modeldiscuss.huggingface.co2021-07-032026-10-07hn
Few-Shot Learning in Practice: GPT-Neo & 'HuggingFace' Accelerated Inference APIhuggingface.co2021-06-042026-10-07hn
NovelAI, a new AI storyteller being built on GPT-Neonovelai.medium.com2021-05-312026-10-07hn
GPT-3's free alternative GPT-Neo is something to be excited aboutventurebeat.com2021-05-152026-10-07hn
GPT-Neo vs. GPT-3: Are Commercialized NLP Models That Much Better?medium.com2021-04-092026-10-07hn
How to run inference with GPT-Neo for freetwitter.com2021-04-052026-10-07hn
GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight modelsgithub.com2021-03-212026-10-07hn
GPT-Neo – Building a GPT-3-sized model, open source and freeeleuther.ai official2021-01-182026-10-07hn

Papers

1 paper

Papers 1

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Similar models

3 models

Similar models

  • gpt2 · 137M
    same size band (S), same task (text-generation), permissive license
  • SmolLM2-135M · 135M
    same size band (S), same task (text-generation), permissive license
  • SmolLM-135M · 135M
    same size band (S), same task (text-generation), permissive license

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:38 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:46 UTC · Summary: 2026-10-07 15:08 UTC · Last verified 2026-10-25

Follow gpt-neo-125m

Watch this model

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.