gpt-neo-1.3B

1.37B parameters, text in, text out, 2K tokens of context.

Verified 2026-10-26 07:27 UTC Rank #447 in LLMs · score 78.1 method v1 Sources 3 Downloads 30d 25.1k · #439 of 619 tracked

Can I run it?0.8GB at INT4Estimatefits a 16 GB cardmethod

Can I use it?MITPermissivelicence file2026-10-26

Do people use it?25.1kdownloads, 30 days#439 of 619 we track · below the median+0 likes, downloads −24% this week326 likesHugging Face2026-10-26

Worth reading

Picked by rule from 15 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Most discussed · 561 HN pts

GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models

The GPT-Neo repository offers an archival implementation of model parallel GPT-3-style models using the mesh-tensorflow library, superseded by GPT-NeoX for GPU efficiency.

Training and inference is officially supported on TPU and should work on GPU as well.
Google colab provides tpu-v8s for free, which should be enough to finetune our models up to GPT3XL (1.5B parameter) sizes.

github.com 2021-03-21 561 HN pts · 127 comments 3k words about the gpt neo family family

Latest · 4 years ago

Hugging Face Transformers – Big Model Inference, Bloom, GPT Neo-X, LongT5 etc.

github.com 2022-06-16 2 HN pts 3.1k words about the gpt neo family family

15 best of 15 tracked

The 2 picks above, plus these 13. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

ArticleGPT-Neo – Building a GPT-3-sized model, open source and freeeleuther.ai · 2021-01-18 · 725 HN pts · official
ArticleFew-Shot Learning in Practice: GPT-Neo & 'HuggingFace' Accelerated Inference APIhuggingface.co · 2021-06-04 · 69 HN pts
ArticleUsing GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learningnlpcloud.io · 2021-11-10 · 24 HN pts
DiscussionH3 – Outperforming GPT-Neo-2.7B with only 2 attention layerstwitter.com · 2023-01-23
ArticleGPT-Neo vs. GPT-3: Are Commercialized NLP Models That Much Better?medium.com · 2021-04-09 · 2 HN pts
ArticleGPT-Neo on a M1 (macOS 12)lazycoder.ro · 2022-02-03 · 5 HN pts
ArticleGPT-3's free alternative GPT-Neo is something to be excited aboutventurebeat.com · 2021-05-15 · 5 HN pts
ArticleGPT-3 open-source alternatives: GPT-Neo and GPT-Jnlpcloud.io · 2021-12-14 · 4 HN pts
ArticleNovelAI, a new AI storyteller being built on GPT-Neonovelai.medium.com · 2021-05-31 · 2 HN pts
ArticlePretrain GPT-Neo for Open Source GitHub Copilot Modeldiscuss.huggingface.co · 2021-07-03
ArticleHow to monitor the performance of a fine-tuned GPT-Neo Model in Productionblog.cerebrium.ai · 2023-02-28
DiscussionFew-shot learning NLP with GPT-J and GPT-Neonews.ycombinator.com · 2021-07-22
DiscussionHow to run inference with GPT-Neo for freetwitter.com · 2021-04-05

Show all 13

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 100 Spaces.

As a service0no API provider listed
On own hardware0quantized builds
Built on13839 fine-tunes · 99 adapters · 0 merges
In apps100+Hugging Face Spaces

Top Spaces running it: Sharathhebbar24/One-stop-for-Open-source-models ♥ 96 · KBaba7/Quant ♥ 51 · auto-cap/MoE-CAP-Dashboard ♥ 35 · FallnAI/Quantize-HF-Models ♥ 19

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 16 GB card at INT4

About 0.8 GB at INT4, 1.6 GB at FP8, 3.3 GB at BF16: weights only, so leave room for the KV cache.

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

Most coverage came in the first two weeks

Teal is the weights (2022-03-02), amber triangles are 10 dated articles (each links to it), grey ticks are repo commits.

What people built on it

138 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

138 derivatives on the Hub, most of them adapters

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

All variants →

Reference

Everything else, folded: the full tables, every source.

Benchmarks

18 rows

Benchmarks 18 rows

measured third-party leaderboard rowauthor-reported number from the model card or eval files (18)No confidence intervals unless the source gave them; settings differ between rows, so compare within a benchmark, not across. Date = the benchmark's own date when it gives one, else when we read it.

Benchmark	Value	Setting / notes	Reported by	Date
author-reported ARC (25-shot)	31.23	Open LLM Leaderboard	model author	2026-10-07
author-reported Avg.	29.44	Open LLM Leaderboard	model author	2026-10-07
author-reported Commonsense QA sets · Chat and follow instructions	71.11% percent	GPT-Neo 1.3B	model author	2026-10-07
author-reported DROP (3-shot)	4.6	Open LLM Leaderboard	model author	2026-10-07
author-reported GSM8K · Reason through hard problems	0.45	Open LLM Leaderboard	model author	2026-10-07
author-reported HellaSwag · Chat and follow instructions	38.66% percent	GPT-Neo 1.3B	model author	2026-10-07
author-reported HellaSwag · Chat and follow instructions	48.47	Open LLM Leaderboard	model author	2026-10-07
author-reported Lambada Acc	57.23% percent	GPT-Neo 1.3B	model author	2026-10-07
author-reported Lambada PPL	7.5	GPT-Neo 1.3B	model author	2026-10-07
author-reported MATH-500 · Reason through hard problems	24.05% percent	GPT-Neo 1.3B	model author	2026-10-07
author-reported MMLU · Reason through hard problems	24.82	Open LLM Leaderboard	model author	2026-10-07
author-reported Pile BPB	0.75	GPT-Neo 1.3B	model author	2026-10-07
author-reported Pile PPL	6.16	GPT-Neo 1.3B	model author	2026-10-07
author-reported PubMedQA	54.40% percent	GPT-Neo 1.3B	model author	2026-10-07
author-reported TruthfulQA · Chat and follow instructions	39.63	Open LLM Leaderboard	model author	2026-10-07
author-reported Wikitext PPL	13.1	GPT-Neo 1.3B	model author	2026-10-07
author-reported Winogrande · Chat and follow instructions	55.01% percent	GPT-Neo 1.3B	model author	2026-10-07
author-reported Winogrande · Chat and follow instructions	56.91	Open LLM Leaderboard	model author	2026-10-07

What the card says

2 cited sentences

Summary generated, cited

GPT-Neo 1.3B is a 1.37B parameter transformer model designed for text generation.^[1]^[2] The model uses the GPTNeoForCausalLM architecture and was trained on the Pile dataset.^[3]^[4]

parameters human
“text-generation”
architectures
“trained on the Pile”

Generated 2026-10-07 15:09 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

3 use cases

What it is good at 3 use cases

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

20 fields

Listed in LLMs

Facts

gpt-neo-1.3B has 1.37B parameters. (safetensors metadata, observed 2026-10-26)
Context length is 2,048 tokens (2K). (config.json max_position_embeddings, observed 2026-10-26)
License: MIT (friction: none). Permissive. (license file, observed 2026-10-26)
Created 2022-03-02, last modified 2024-01-31. (HF API, observed 2026-10-26)

Pipeline tag	text-generation
Library	transformers
Architectures	GPTNeoForCausalLM
Model type	gpt_neo
Params by dtype	F32: 1,315,575,808 · U8: 50,331,648
Size band	S
MoE	no
Context length	2,048 (config.json max_position_embeddings)
Languages	en
Papers	arXiv:2101.00027
License	MIT (mit)
Hub snapshot	2026-10-26

Servability

vLLM · SGLang

Servability

vLLM: not in registry · 4b1c04e76fe9ac3ca8b7e541dd423e7d5f38dc60SGLang: not in registry · 60d941293e3af27a12c6eb64e2c3e2e1a17f100f

vLLM registry (4b1c04e76fe9ac3ca8b7e541dd423e7d5f38dc60) · SGLang registry (60d941293e3af27a12c6eb64e2c3e2e1a17f100f) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for gpt-neo-1.3B: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)	3.3 GB · Edge (<4 GB)
FP8 (1 byte/param)	1.6 GB · Edge (<4 GB)
INT4 (0.5 bytes/param)	0.8 GB · Edge (<4 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.

All variants

40 variants

Lineage

Original model Family: EleutherAI/gpt-neo 138 derivatives on the Hub

finetune: 39adapter: 99

Top variants (40 shown, ranked by downloads)

Variant	Relation	Quant	Params	Band	License	Author	Likes	Downloads
postbot/gpt-neo-1.3B-emailgen	finetune	—	1.42B	S	apache-2.0	postbot	4	11k
Kimargin/GPT-NEO-1.3B-wiki	finetune	—	1.32B	S	apache-2.0	Kimargin	1	564
JakeTurner616/Adonalsium-gpt-neo-1.3B	finetune	—	1.42B	S	mit	JakeTurner616	1	455
KimByeongSu/gpt-neo-1.3B_LAMA_TREx_finetuning_aeda	finetune	—	—	—	mit	KimByeongSu	0	401
ArashMarandi/Llama-2-7b-chat-hf	finetune	—	1.32B	S	mit	ArashMarandi	0	400
KimByeongSu/gpt-neo-1.3B_LAMA_TREx_finetuning_eda	finetune	—	—	—	mit	KimByeongSu	0	387
dod-llms/gptneo_1.3B_medical_multitransPT	finetune	—	1.32B	S	cc-by-nc-4.0	dod-llms	2	262
KimByeongSu/gpt-neo-1.3B-cs-finetuning-7	finetune	—	1.32B	S	mit	KimByeongSu	0	325
sirine21/gptneo-finetuned-seo	finetune	—	1.32B	S	mit	sirine21	0	306
KimByeongSu/gpt-neo-1.3B_LAMA_TREx_finetuning_MAGNET_valid	finetune	—	—	—	mit	KimByeongSu	0	291
ICT3214-Group5/MD5_gpt_neo_L_v1.0.0	finetune	—	1.32B	S	mit	ICT3214-Group5	0	217
MyMoodAI/basicmood	adapter	—	—	—	—	MyMoodAI	0	215
Ben10x/gpt-medmentions	finetune	—	1.32B	S	mit	Ben10x	0	215
ishikaibm/peft_ft_random	adapter	—	—	—	mit	ishikaibm	0	150
doguilmak/inferencevision-gpt-neo-1.3B	finetune	—	1.32B	S	mit	doguilmak	0	136
Mrigendra100/XpathGenerator	finetune	—	1.32B	S	—	Mrigendra100	2	30
Torrchy/fine-tuned-gpt-neo	finetune	—	1.32B	S	mit	Torrchy	0	110
KimByeongSu/neo-1.3B-fine-same	finetune	—	1.32B	S	mit	KimByeongSu	0	109
gandhiraketla277/gpt-neo-1.3b-dolly15k-lora	adapter	—	—	—	apache-2.0	gandhiraketla277	0	104
raimondskrauklis/gpt-neo-1.3b-code-conversation	finetune	—	1.32B	S	apache-2.0	raimondskrauklis	0	96
kk-aivio/4311cae5-aba2-4421-bbcb-8c3f054bcc76	adapter	—	—	—	mit	kk-aivio	0	83
mia-llm/gpt-neo-1.3B-wikitext2raw-roya	finetune	—	1.32B	S	—	mia-llm	0	81
ishikaibm/peft_ft_fl_mod_dep	adapter	—	—	—	mit	ishikaibm	0	80
parzival911/gpt-neo-lora	adapter	—	—	—	—	parzival911	0	79
KimByeongSu/gpt-neo-1.3B_LAMA_TREx_finetuning_MAGNET	finetune	—	—	—	mit	KimByeongSu	0	70
betobragajr/zabbix-model	finetune	—	1.32B	S	mit	betobragajr	0	70
tomhata/ul-wk2-gpt-neo-1.3B	finetune	—	1.32B	S	—	tomhata	0	67
KimByeongSu/gpt-neo-1.3B_LAMA_TREx_finetuning	finetune	—	—	—	mit	KimByeongSu	0	66
mia-llm/gpt-neo-1.3B-AGnews-roya	finetune	—	1.32B	S	—	mia-llm	0	64
raimondskrauklis/gpt-neo-1.3b-lumi-conversational	finetune	—	1.32B	S	apache-2.0	raimondskrauklis	0	64
KimByeongSu/gpt-neo-1.3B-cs-finetuning-filtered-7	finetune	—	1.32B	S	mit	KimByeongSu	0	63
mia-llm/gpt-neo-1.3B-xsum-roya	finetune	—	1.32B	S	—	mia-llm	0	62
KimByeongSu/gpt-neo-1.3B-lama	finetune	—	1.32B	S	mit	KimByeongSu	0	55
TheSky0908/gpt-neo-1.3B-toxicity-surrogate	finetune	—	1.32B	S	apache-2.0	TheSky0908	0	55
KimByeongSu/gpt-neo-1.3B-cs-finetuning-500-1	finetune	—	1.32B	S	mit	KimByeongSu	0	54
mia-llm/autotrain-gptneo-13b-sft-slm	finetune	—	1.32B	S	other	mia-llm	0	51
therealcyberlord/magicwand-gptneo-1.3b	adapter	—	—	—	—	therealcyberlord	0	44
error577/d812ccbf-2b40-4bb7-a951-21c412b54967	adapter	—	—	—	mit	error577	0	41
vmpsergio/b4ec4945-1b25-4aae-b652-e3c9fb8c5856	adapter	—	—	—	mit	vmpsergio	0	39
adalib/torchrec-data-gpt-neo-1.3B-prefix	adapter	—	—	—	—	adalib	0	35

Repository changelog

28 commits

Repository changelog 28 commits

Date	Commit	SHA
2024-01-31	Adding Evaluation Results (#7)	dbe59a7
2023-07-09	Adding `safetensors` variant of this model (#6)	8282180
2023-06-21	Update README.md	9d51b51
2022-12-08	Update README.md	0f35a20
2021-12-31	Update README.md	7971745
2021-12-31	Updated tags to correctly link with the Pile	957b2c0
2021-09-11	Update README.md	2e854e0
2021-09-11	Update README.md	875dac1
2021-07-04	add flax model	a4a1108
2021-05-20	Update README.md	60168e2
2021-05-20	Update README.md	26e74b2
2021-05-18	updated citation info	4ca242d
2021-05-04	updated LFS git attributes	9e2547e
2021-05-04	Addition of Rust model	9c768b0
2021-04-12	Update README.md	a7c07bf
2021-04-06	Update model max length	a6259a0
2021-04-04	Update README.md	b873dab
2021-04-02	Update README.md	6184244
2021-03-31	updated model card title	3c81723
2021-03-31	Update README.md	fd3a60d
2021-03-31	Update config.json	22761ae
2021-03-30	Sample	224decc
2021-03-30	Update model card	12f02fa
2021-03-30	Update model card	15b02c1
2021-03-30	Update model card	5b0388c
2021-03-30	Add model card	63d9f33
2021-03-30	add files	765fd74
2021-03-30	initial commit	a454351

Articles and coverage

15 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Official2

GPT-Neo – Building a GPT-3-sized model, open source and free

eleuther.ai2021-01-18officialHN front pageabout the gpt neo familyHN 725 pts · 252 comments

Few-Shot Learning in Practice: GPT-Neo & 'HuggingFace' Accelerated Inference API

huggingface.co2021-06-04authorityabout the gpt neo familyHN 69 pts · 3 comments1,201 words

GPT-Neo enables few-shot learning for NLP tasks by requiring only 3-4 examples to achieve good results, as it is about 60x smaller than GPT-3. The 🤗 Accelerated Inference API hosts this 2.7B parameter model and offers up to 100x speedup compared to standard Transformers deployments. Engineers can control generation randomness via temperature and stop sequences using simple API calls.written, reviewed, cited

GPT-Neo checkpoints are trained exclusively on the Pile dataset, so performance degrades on text distributions different from its training corpus.
The Accelerated Inference API supports inference on over 10,000 publicly available models or private models via simple API calls.
Regulations like GDPR require that users receive an explanation for automatic decisions made about them by the system.

Analysis8

GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models

github.com2021-03-21authorityHN front pageabout the gpt neo familyHN 561 pts · 127 comments2,997 words

The GPT-Neo repository offers an archival implementation of model parallel GPT-3-style models using the mesh-tensorflow library, superseded by GPT-NeoX for GPU efficiency. As of August 2021, the code is no longer maintained. Two pretrained models trained on The Pile are available in 1.3B and 2.7B sizes.written, reviewed, cited

Training and inference is officially supported on TPU and should work on GPU as well.
Google colab provides tpu-v8s for free, which should be enough to finetune our models up to GPT3XL (1.5B parameter) sizes.
Some users have reported having difficulty getting MTF to recognize their GPUs.

Using GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learning

nlpcloud.io2021-11-10about the gpt neo familyHN 24 pts · 1 comments

Hugging Face Transformers – Big Model Inference, Bloom, GPT Neo-X, LongT5 etc.

github.com2022-06-16authorityabout the gpt neo familyHN 2 pts · 0 comments3,146 words

GPT-Neo vs. GPT-3: Are Commercialized NLP Models That Much Better?

medium.com2021-04-09about the gpt neo familyHN 2 pts · 2 comments

GPT-3's free alternative GPT-Neo is something to be excited about

venturebeat.com2021-05-15about the gpt neo familyHN 5 pts · 0 comments

GPT-Neo on a M1 (macOS 12)

lazycoder.ro2022-02-03about the gpt neo familyHN 5 pts · 0 comments557 words

GPT-3 open-source alternatives: GPT-Neo and GPT-J

nlpcloud.io2021-12-14about the gpt neo familyHN 4 pts · 0 comments

NovelAI, a new AI storyteller being built on GPT-Neo

novelai.medium.com2021-05-31about the gpt neo familyHN 2 pts · 1 comments

All coverage tracked for this model (15) — append-only, nothing is ever removed

Title	Source	Published	First seen	Signals
How to monitor the performance of a fine-tuned GPT-Neo Model in Production	blog.cerebrium.ai	2023-02-28	2026-10-07	hn
H3 – Outperforming GPT-Neo-2.7B with only 2 attention layers	twitter.com	2023-01-23	2026-10-07	hn
Hugging Face Transformers – Big Model Inference, Bloom, GPT Neo-X, LongT5 etc.	github.com	2022-06-16	2026-10-07	hn
GPT-Neo on a M1 (macOS 12)	lazycoder.ro	2022-02-03	2026-10-07	hn
GPT-3 open-source alternatives: GPT-Neo and GPT-J	nlpcloud.io	2021-12-14	2026-10-07	hn
Using GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learning	nlpcloud.io	2021-11-10	2026-10-07	hn
Few-shot learning NLP with GPT-J and GPT-Neo	news.ycombinator.com	2021-07-22	2026-10-07	hn
Pretrain GPT-Neo for Open Source GitHub Copilot Model	discuss.huggingface.co	2021-07-03	2026-10-07	hn
Few-Shot Learning in Practice: GPT-Neo & 'HuggingFace' Accelerated Inference API	huggingface.co	2021-06-04	2026-10-07	hn
NovelAI, a new AI storyteller being built on GPT-Neo	novelai.medium.com	2021-05-31	2026-10-07	hn
GPT-3's free alternative GPT-Neo is something to be excited about	venturebeat.com	2021-05-15	2026-10-07	hn
GPT-Neo vs. GPT-3: Are Commercialized NLP Models That Much Better?	medium.com	2021-04-09	2026-10-07	hn
How to run inference with GPT-Neo for free	twitter.com	2021-04-05	2026-10-07	hn
GPT Neo: open-source GPT model, with pretrained 1.3B & 2.7B weight models	github.com	2021-03-21	2026-10-07	hn
GPT-Neo – Building a GPT-3-sized model, open source and free	eleuther.ai official	2021-01-18	2026-10-07	hn

Papers

1 paper

Papers 1

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

The Pile: An 800GB Dataset of Diverse Text for Language Modeling cited in card

Leo Gao, Stella Biderman, Sid Black +9 · 2020-12-31 · PDF · HF paper page (10 upvotes)

Similar models

3 models

Similar models

Llama-3.2-1B-Instruct · 1.24B
same size band (S), same task (text-generation), more providers
Llama-3.2-1B · 1.24B
same size band (S), same task (text-generation), more providers
Qwen2.5-1.5B-Instruct · 1.54B
same size band (S), same task (text-generation), more providers, permissive license

Sources

3 sources

Sources

Hub snapshot: 2026-10-26 · HF API: 2026-10-26 07:20 UTC · Providers: 2026-10-26 07:26 UTC · Articles: 2026-10-26 07:27 UTC · Summary: 2026-10-07 15:09 UTC · Last verified 2026-10-26 07:27 UTC

Follow gpt-neo-1.3B

Watch this model

/feeds/models/EleutherAI--gpt-neo-1.3B.xml(RSS 2.0) · every change to gpt-neo-1.3B
/feeds/families/EleutherAI--gpt-neo.xml(RSS 2.0) · every change across the gpt-neo family

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.