Models › gpt-j family

EleutherAI Apache 2.0 out 5 years ago

Text in, text out, 2K tokens of context.

Verified 2026-10-25 Rank #194 in LLMs · score 89.5 method v1 Sources 3 Downloads 30d 126.2k · #291 of 619 tracked

Can I use it?Apache 2.0Permissive; patent grantlicence file2026-10-24
Do people use it?126.2kdownloads, 30 days#291 of 619 we track · top 47%+3 likes, downloads −46% this week1,536 likesHugging Face2026-10-24

Worth reading

Picked by rule from 31 tracked pieces: the lab's own post, the most shared, a field report, the latest.

Most discussed · 636 HN pts

GPT-J-6B – A 6 billion parameter, autoregressive text generation model

Mesh-Transformer-JAX implements model parallel transformers in JAX using xmap/pjit operators, similar to Megatron-LM, and supports scaling up to approximately 40B parameters on TPUv3s.

  • The device_* scripts only work on a v3-8 and not on larger pods.
  • The Megatron-11B model provides no comparable metrics, and several implementations using the released weights do not reproduce the generation quality and evaluations.
github.com 2021-07-04 636 HN pts · 146 comments 1.3k words
Field report

Fun and dystopia with AI-based code generation using GPT-J-6B

GPT-J is a 6 billion parameter model by EleutherAI that outperforms GPT-3 on code generation tasks despite its smaller size.

  • The model outputs the alternate solution using bitwise operators for the is_even function.
minimaxir.com 2021-06-14 177 HN pts · 40 comments 3.3k words

25 best of 31 tracked

The 3 picks above, plus these 22. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

  1. ArticleGPT-J-6B: 6B JAX-Based Transformerarankomatsuzaki.wordpress.com · 2021-06-04 · 144 HN pts
  2. ArticleGPT-J: GPT-3 Democratizedp3r.one · 2021-07-04 · 118 HN pts
  3. ArticleTuning and Testing Llama 2, Flan-T5, and GPT-J with LoRA, Sematic, and Gradiosematic.dev · 2023-07-26 · 98 HN pts
  4. ArticleShow HN: Exafunction NLP – Run models like GPT-J for 6x cheaper than OpenAIexafunction.com · 2021-09-28 · 45 HN pts
  5. ArticleChatGPT-J: The Privacy-First, Self-Hosted Chatbot Built on GPT-J's Powerful AIcolab.research.google.com · 2023-03-10 · 54 HN pts
  6. ArticleGPT-J is self-hosted open-source analog of GPT-3: how to run in Dockertracklify.com · 2022-03-27 · 39 HN pts
  7. ArticleGPT-J, an open-source alternative to GPT-3narrativa.com · 2022-12-14 · 15 HN pts
  8. ArticlePhallocentricity in GPT-J's stratified ontologylesswrong.com · 2024-02-24
  9. ArticleUsing GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learningnlpcloud.io · 2021-11-10
  10. DiscussionGPT-J for text generation: hardware requirementsnews.ycombinator.com · 2021-07-05
  11. ArticleAn imaginary (multiversal) web browser based on GPT-3/Codex/GPT-jsemiosis.github.io · 2021-09-11
  12. ArticleOODA Loop – Open-Source Natural Language Processing: EleutherAI's GPT-Joodaloop.com · 2022-03-16
  13. ArticleGPT-J and GPT-3 combined with Pen.el (libre copilot for Emacs)mullikine.github.io · 2021-08-03
  14. ArticleNSFW GPT-J Based Chatbotsyourway.ai · 2022-06-15
  15. ArticleRun your own ChatGPT(EleutherAI/GPT-j-6B) on your MacBook M1bbarrows.com · 2023-02-24
  16. ArticleDemo – Text generation with EleutherAI's GPT-J-6B modelapp.baseten.co · 2022-04-29
  17. ArticleGPT-J-6B6b.eleuther.ai · 2021-06-10 · official
  18. DiscussionI grabbed a paragraph from Wikipedia and had GPT-J come up with a ? about itnews.ycombinator.com · 2021-08-17
  19. DiscussionGenji-Python-6B, a GPT-J 6B model for generating Python codetwitter.com · 2021-08-06
  20. ArticleHow to Harness the Predictive Power of GPT-J | Cerebrascerebras.ai · undated
  21. ArticleGPT-J-6B: An Introduction to the Largest Open Source GPT Model | Forefrontweb.archive.org · undated
  22. ArticleGPT-J - ZigLlamadocs.cognisoc.com · undated

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 3 quantized builds and 100 Spaces.

  • As a service0no API provider listed
  • On own hardware3quantized builds
  • Built on5321 fine-tunes · 32 adapters · 0 merges
  • In apps100+Hugging Face Spaces

Top Spaces running it: h2oai/h2ogpt-chatbot ♥ 295 · h2oai/h2ogpt-chatbot2 ♥ 167 · Sharathhebbar24/One-stop-for-Open-source-models ♥ 96 · bigscience/bloom-book ♥ 72

The story so far

Weights, every dated article, and repo commits on one rail.

Most coverage came in the first two weeks

Teal is the weights (2022-03-02), amber triangles are 10 dated articles (each links to it), grey ticks are repo commits.

JulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2023-06-21: 1 repo commit2023-03-29: 1 repo commit2022-10-25: 1 repo commit2022-03-15: 1 repo commit2022-03-08: 1 repo commit2022-01-11: 1 repo commit2021-11-23: 1 repo commit2021-09-22: 2 repo commits2021-08-31: 1 repo commit2021-08-25: 1 repo commit2021-08-23: 1 repo commit2021-08-18: 1 repo commit2021-08-09: 1 repo commit2021-08-06: 4 repo commits2021-08-05: 7 repo commitsGPT-J-6B – A 6 billion parameter, autoregressive text generation model · 2021-07-04Textsynth: Bellard's free GPT-NeoX-20B, GPT-J playground and paid API · 2022-04-27GPT-J-6B: 6B JAX-Based Transformer · 2021-06-04Fun and dystopia with AI-based code generation using GPT-J-6B · 2021-06-14GPT-J: GPT-3 Democratized · 2021-07-04Tuning and Testing Llama 2, Flan-T5, and GPT-J with LoRA, Sematic, and Gradio · 2023-07-26Show HN: Exafunction NLP – Run models like GPT-J for 6x cheaper than OpenAI · 2021-09-28ChatGPT-J: The Privacy-First, Self-Hosted Chatbot Built on GPT-J's Powerful AI · 2023-03-10GPT-J is self-hosted open-source analog of GPT-3: how to run in Docker · 2022-03-27GPT-J, an open-source alternative to GPT-3 · 2022-12-14Weights 2022-03-02 · 2022-03-02Weights 2022-03-02now
JulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2023-06-21: 1 repo commit2023-03-29: 1 repo commit2022-10-25: 1 repo commit2022-03-15: 1 repo commit2022-03-08: 1 repo commit2022-01-11: 1 repo commit2021-11-23: 1 repo commit2021-09-22: 2 repo commits2021-08-31: 1 repo commit2021-08-25: 1 repo commit2021-08-23: 1 repo commit2021-08-18: 1 repo commit2021-08-09: 1 repo commit2021-08-06: 4 repo commits2021-08-05: 7 repo commitsGPT-J-6B – A 6 billion parameter, autoregressive text generation model · 2021-07-04Textsynth: Bellard's free GPT-NeoX-20B, GPT-J playground and paid API · 2022-04-27GPT-J-6B: 6B JAX-Based Transformer · 2021-06-04Fun and dystopia with AI-based code generation using GPT-J-6B · 2021-06-14GPT-J: GPT-3 Democratized · 2021-07-04Tuning and Testing Llama 2, Flan-T5, and GPT-J with LoRA, Sematic, and Gradio · 2023-07-26Show HN: Exafunction NLP – Run models like GPT-J for 6x cheaper than OpenAI · 2021-09-28ChatGPT-J: The Privacy-First, Self-Hosted Chatbot Built on GPT-J's Powerful AI · 2023-03-10GPT-J is self-hosted open-source analog of GPT-3: how to run in Docker · 2022-03-27GPT-J, an open-source alternative to GPT-3 · 2022-12-14Weights 2022-03-02 · 2022-03-02Weights 2022-03-02now

What people built on it

56 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

56 derivatives on the Hub, most of them adapters

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

3 quantized (Hugging Face model tree)21 fine-tunes (Hugging Face model tree)32 adapters (Hugging Face model tree)3 quantized21 fine-tunes32 adapters
3 quantized (Hugging Face model tree)21 fine-tunes (Hugging Face model tree)32 adapters (Hugging Face model tree)3 quantized21 fine-tunes32 adapters

All variants →

Reference

Everything else, folded: the full tables, every source.

Benchmarks

5 rows

Benchmarks 5 rows

measured third-party leaderboard rowauthor-reported number from the model card or eval files (5)No confidence intervals unless the source gave them; settings differ between rows, so compare within a benchmark, not across. Date = the benchmark's own date when it gives one, else when we read it.
BenchmarkValueSetting / notesReported byDate
author-reported Commonsense QA sets · Chat and follow instructions76.5% percentGPT-J 6Bmodel author2026-10-07
author-reported HellaSwag · Chat and follow instructions66.1% percentGPT-J 6Bmodel author2026-10-07
author-reported LAMBADA Acc ↑69.7% percentGPT-J 6Bmodel author2026-10-07
author-reported LAMBADA PPL ↓3.99 otherGPT-J 6Bmodel author2026-10-07
author-reported Winogrande · Chat and follow instructions65.3% percentGPT-J 6Bmodel author2026-10-07

Use-case evidence

1 use case

What it is good at 1 use case

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

17 fields

Listed in LLMs

Facts

Pipeline tagtext-generation
Librarytransformers
ArchitecturesGPTJForCausalLM
Model typegptj
MoEno
Context length2,048 (config.json max_position_embeddings)
Languagesen
PapersarXiv:2104.09864, arXiv:2101.00027
LicenseApache 2.0 (apache-2.0)
Hub snapshot2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · GPTJForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · GPTJForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

All variants

40 variants

Lineage

Original model Family: EleutherAI/gpt-j 56 derivatives on the Hub

finetune: 21quantized: 3adapter: 32

Top variants (40 shown, ranked by downloads)

Relation
Quant
40 rows
Variant Relation Quant Params Band License Author Likes Downloads
bertin-project/bertin-gpt-j-6B finetune — 6.06B M apache-2.0 bertin-project 19 34.9k
allura-org/GPT-J-6b-Disco-Elysium finetune — 6.05B M apache-2.0 allura-org 5 2.4k
flytech/gpt-j-6b-devchat finetune — — — apache-2.0 flytech 0 2k
KimByeongSu/gpt-j-6b_LAMA_TREx_finetuning finetune — — — apache-2.0 KimByeongSu 0 1.7k
YALCINKAYA/opsgenius_j quantized — 6.23B M apache-2.0 YALCINKAYA 0 1.3k
monsterapi/Gptj-6b_alpaca-gpt4 adapter — — — — monsterapi 0 1.1k
OpenVINO/gpt-j-6b-int4-ov quantized int4 — — apache-2.0 OpenVINO 0 711
grugging/gpt-j-6b-grug finetune — 6.05B M apache-2.0 grugging 0 680
Uppaal/gpt-j-ProFS-toxicity finetune — 6.05B M mit Uppaal 0 490
OpenVINO/gpt-j-6b-int8-ov quantized int8 — — apache-2.0 OpenVINO 0 482
OpenVINO/gpt-j-6b-fp16-ov finetune -fp16 — — apache-2.0 OpenVINO 0 403
andstor/gpt-j-6B-smart-contract finetune — 6.17B M apache-2.0 andstor 5 6
henriceriocain/HenriAI finetune — — — apache-2.0 henriceriocain 0 221
furiosa-ai-dev/gpt-j-6b-FP8-MLPerf finetune -FP8 — — apache-2.0 furiosa-ai-dev 0 200
txw123/scamgptj-large adapter — — — — txw123 0 153
monsterapi/gptj_6b_WizardLMEvolInstruct70k adapter — — — apache-2.0 monsterapi 0 131
ksee/testgptj adapter — — — — ksee 0 123
XanderJC/gptj-sft-tldr adapter — — — apache-2.0 XanderJC 0 122
allura-forge/phi-j-6b-edited finetune — 6.05B M apache-2.0 allura-forge 0 122
Holarissun/trl_rm_tldr_gptj adapter — — — apache-2.0 Holarissun 1 61
sanduntg/lora-dolly adapter — — — apache-2.0 sanduntg 0 107
ikyuarqie/GPTJodoo100 finetune — 6.05B M other ikyuarqie 0 91
Aung2024/gpt2-a1-model finetune — — — — Aung2024 0 83
txw123/scamgptj adapter — — — — txw123 1 32
ikyuarqie/gptjodoo4000i finetune — 6.05B M other ikyuarqie 0 80
ping98k/gpt-j-6b-thinking-no-special-sft-lora adapter — — — apache-2.0 ping98k 0 72
ping98k/gpt-j-6b-thinking-sft-lora adapter — — — apache-2.0 ping98k 0 63
Holarissun/gptj6b-aisft-static-seqsampler-subset5000 adapter — — — apache-2.0 Holarissun 0 56
allura-forge/phi-j-6b finetune — 6.05B M apache-2.0 allura-forge 0 50
Bandika/DollyV1 adapter — — — — Bandika 0 44
mehta77/dolly-lora adapter — — — — mehta77 0 43
MHGanainy/gpt-J-lora-ecthr adapter — — — apache-2.0 MHGanainy 0 41
Bandika/DollyV2 adapter — — — — Bandika 0 36
Holarissun/gptj6b-aisft-hh-seqsampler-subset2000 adapter — — — apache-2.0 Holarissun 0 36
Holarissun/gptj6b-aisft-hh-seqsampler-subset60000 adapter — — — apache-2.0 Holarissun 0 36
Holarissun/gptj6b-aisft-giga-seq-subset100000 adapter — — — apache-2.0 Holarissun 0 36
mehta77/dolly-lora_20240104 adapter — — — — mehta77 0 33
mehta77/dolly-lora_20240104_b adapter — — — — mehta77 0 33
Holarissun/gptj6b-aisft-hh-randsampler-subset2000 adapter — — — apache-2.0 Holarissun 0 33
anusmriti298/dolly_6B_298A_model1 adapter — — — — anusmriti298 0 31

Repository changelog

25 commits

Repository changelog 25 commits

DateCommitSHA
2023-06-21 Update README.md 47e1693
2023-03-29 updated the use section f98c709
2022-10-25 Fix dataset name (#4) 6e35e21
2022-03-15 add TF model 918ad37
2022-03-08 add flax model a2044a6
2022-01-11 Update tokenizer_config.json 4a035bd
2021-11-23 Update README.md bac4a8a
2021-09-22 fix model path ae77a55
2021-09-22 add extra tokens d3109d2
2021-08-31 correct checkpoint d2ee43d
2021-08-25 Update README.md a05826a
2021-08-23 adapt model weights a16214f
2021-08-18 Restore model functionality 35405f4
2021-08-09 Update config.json 9dd387f
2021-08-06 Update README.md b1bf2ae
2021-08-06 Update README.md 7cb5428
2021-08-06 Update README.md 8a44d5a
2021-08-06 Update README.md d3d2956
2021-08-05 Corrected minor errors 4824bed
2021-08-05 Update to GPT2 style 2208576
2021-08-05 upload 6B weights 658edeb
2021-08-05 Update README.md 674a405
2021-08-05 Update README.md e8ee971
2021-08-05 Create README.md 7210057
2021-08-05 initial commit f5ee4a9

Articles and coverage

31 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Analysis10

github.com2021-07-04authorityHN front pageHN 636 pts · 146 comments1,329 words
Mesh-Transformer-JAX implements model parallel transformers in JAX using xmap/pjit operators, similar to Megatron-LM, and supports scaling up to approximately 40B parameters on TPUv3s. The library includes an experimental ZeRo style sharding version and released a 6 billion parameter autoregressive text generation model trained on The Pile under the Apache License version 2.0. Using a TPU v3-8, users can fine-tune the model at a rate of ~5000 tokens/second, though the v1 models specifically require jax==0.2.12.written, reviewed, cited
  • The device_* scripts only work on a v3-8 and not on larger pods.
  • The Megatron-11B model provides no comparable metrics, and several implementations using the released weights do not reproduce the generation quality and evaluations.
textsynth.com2022-04-27HN front pageabout the gpt j familyHN 263 pts · 134 comments51 words
arankomatsuzaki.wordpress.com2021-06-04HN front pageHN 144 pts · 79 comments1,362 words
GPT-J-6B is a 6 billion parameter JAX-based transformer language model that performs nearly on par with 6.7B GPT-3 on zero-shot tasks. The model was trained on 400B tokens from The Pile dataset containing 800GB text with an attention head dimension of 256. Training took roughly five weeks on a TPU v3-256 pod, achieving 5.4 PFLOPs of efficiency.written, reviewed, cited
  • Efficient attention mechanisms were not used for simplicity as they would not have significantly improved throughput at this scale.
minimaxir.com2021-06-14HN front pageHN 177 pts · 40 comments3,302 words
GPT-J is a 6 billion parameter model by EleutherAI that outperforms GPT-3 on code generation tasks despite its smaller size. Trained on The Pile, which has a high weight of GitHub and Stack Overflow input, the model often generates correct but convoluted Python code. Running the model requires a TPU to generate text at a reasonable speed.written, reviewed, cited
  • The model outputs the alternate solution using bitwise operators for the is_even function.
p3r.one2021-07-04HN front pageabout the gpt j familyHN 118 pts · 39 comments
sematic.dev2023-07-26about the gpt j familyHN 98 pts · 22 comments24 words
exafunction.com2021-09-28about the gpt j familyHN 45 pts · 19 comments74 words
colab.research.google.com2023-03-10about the gpt j familyHN 54 pts · 9 comments1,036 words
tracklify.com2022-03-27about the gpt j familyHN 39 pts · 7 comments
narrativa.com2022-12-14about the gpt j familyHN 15 pts · 5 comments992 words
All coverage tracked for this model (31) — append-only, nothing is ever removed
TitleSourcePublishedFirst seenSignals
How to Harness the Predictive Power of GPT-J | Cerebrascerebras.ai—2026-10-07exa
GPT-J-6B: An Introduction to the Largest Open Source GPT Model | Forefrontweb.archive.org—2026-10-07exa
GPT-J - ZigLlamadocs.cognisoc.com—2026-10-07exa
Cerebras Makes It Easy to Harness the Predictive Power of GPT-Jcerebras.ai—2026-10-07exa
Phallocentricity in GPT-J's stratified ontologylesswrong.com2024-02-242026-10-07hn
Tuning and Testing Llama 2, Flan-T5, and GPT-J with LoRA, Sematic, and Gradiosematic.dev2023-07-262026-10-07hn
ChatGPT-J: The Privacy-First, Self-Hosted Chatbot Built on GPT-J's Powerful AIcolab.research.google.com2023-03-102026-10-07hn
Run your own ChatGPT(EleutherAI/GPT-j-6B) on your MacBook M1bbarrows.com2023-02-242026-10-07hn
GPT-J, an open-source alternative to GPT-3narrativa.com2022-12-142026-10-07hn
NSFW GPT-J Based Chatbotsyourway.ai2022-06-152026-10-07hn
Demo – Text generation with EleutherAI's GPT-J-6B modelapp.baseten.co2022-04-292026-10-07hn
Textsynth: Bellard's free GPT-NeoX-20B, GPT-J playground and paid APItextsynth.com2022-04-272026-10-07hn
GPT-J is self-hosted open-source analog of GPT-3: how to run in Dockertracklify.com2022-03-272026-10-07hn
OODA Loop – Open-Source Natural Language Processing: EleutherAI's GPT-Joodaloop.com2022-03-162026-10-07hn
Darkspur/GPT-Jgithub.com2022-02-102026-10-07exa
Using GPT-J and GPT-Neo, GPT-3 open-source alternatives, with few-shot learningnlpcloud.io2021-11-102026-10-07hn
Show HN: Exafunction NLP – Run models like GPT-J for 6x cheaper than OpenAIexafunction.com2021-09-282026-10-07hn
An imaginary (multiversal) web browser based on GPT-3/Codex/GPT-jsemiosis.github.io2021-09-112026-10-07hn
I grabbed a paragraph from Wikipedia and had GPT-J come up with a ? about itnews.ycombinator.com2021-08-172026-10-07hn
Checking out a 6B parameter GPT model, GPT-J, from Eleuther AI [video]youtube.com2021-08-092026-10-07hn
Genji-Python-6B, a GPT-J 6B model for generating Python codetwitter.com2021-08-062026-10-07hn
GPT-J and GPT-3 combined with Pen.el (libre copilot for Emacs)mullikine.github.io2021-08-032026-10-07hn
GPT-J for text generation: hardware requirementsnews.ycombinator.com2021-07-052026-10-07hn
GPT-J: GPT-3 Democratizedp3r.one2021-07-042026-10-07hn
GPT-J-6B – A 6 billion parameter, autoregressive text generation modelgithub.com2021-07-042026-10-07hn
GPT-J-6B – Most advanced GPT-3 Alternative for AI Text Generationyoutube.com2021-06-172026-10-07hn
GPT-J 6B on GPUs (through HuggingFace PR)news.ycombinator.com2021-06-162026-10-07hn
Fun and dystopia with AI-based code generation using GPT-J-6Bminimaxir.com2021-06-142026-10-07hn, exa
GPT-J-6B6b.eleuther.ai official2021-06-102026-10-07hn
GPT-J-6B — benchmarks, pricing & hardware fit · Model Beatsmodelbeats.com2021-06-092026-10-07exa
GPT-J-6B: 6B JAX-Based Transformerarankomatsuzaki.wordpress.com2021-06-042026-10-07hn, exa

Papers

2 papers

Papers 2

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Similar models

3 models

Similar models

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:36 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:04 UTC · Last verified 2026-10-25

Follow gpt-j-6b

Watch this model

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.