Up to date
EleutherAI Apache 2.0 out 5 years ago
20.7B parameters, text in, text out, 2K tokens of context.
Verified 2026-10-25 Rank #291 in LLMs · score 84.2 method v1 Sources 3 Downloads 30d 249.2k · #234 of 619 tracked
Picked by rule from 19 tracked pieces: the lab's own post, the most shared, a field report, the latest.
GPT-NeoX-20B, a 20 billion parameter model trained using the GPT-NeoX framework on CoreWeave GPUs, becomes available for download on February 9, 2022.
The rustformers/llm repository is archived due to a lack of time and resources for continued development.
The 4 picks above, plus these 15. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.
Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.
No API provider serves it, so its downloads are people running it themselves: 9 quantized builds and 100 Spaces.
Top Spaces running it: olivierdehaene/chat-llm-streaming ♥ 336 · h2oai/h2ogpt-chatbot ♥ 295 · h2oai/h2ogpt-chatbot2 ♥ 167 · tomg-group-umd/lm-watermarking ♥ 100
Weights only, an estimate: leave room for the KV cache. Method
About 12 GB at INT4, 25 GB at FP8, 50 GB at BF16: weights only, so leave room for the KV cache.
Estimate params × bytes per param × 1.2 · how we estimate
Weights, every dated article, and repo commits on one rail.
Teal is the weights (2022-04-07), amber triangles are 6 dated articles (each links to it), grey ticks are repo commits.
44 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.
From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.
Everything else, folded: the full tables, every source.
| Benchmark | Value | Setting / notes | Reported by | Date |
|---|---|---|---|---|
| author-reported ARC (25-shot) | 45.73 | Open LLM Leaderboard | model author | 2026-10-07 |
| author-reported ARC Challenge · Chat and follow instructions | 0.380 ± 0.014 | Zero-shot | model author | 2026-10-07 |
| author-reported Avg. | 36.02 | Open LLM Leaderboard | model author | 2026-10-07 |
| author-reported Commonsense QA sets · Chat and follow instructions | 0.779 ± 0.010 | Zero-shot | model author | 2026-10-07 |
| author-reported Commonsense QA sets · Chat and follow instructions | 0.259 ± 0.004 | Zero-shot | model author | 2026-10-07 |
| author-reported DROP (3-shot) | 5.04 | Open LLM Leaderboard | model author | 2026-10-07 |
| author-reported GSM8K · Reason through hard problems | 2.43 | Open LLM Leaderboard | model author | 2026-10-07 |
| author-reported HellaSwag · Chat and follow instructions | 73.45 | Open LLM Leaderboard | model author | 2026-10-07 |
| author-reported MMLU · Reason through hard problems | 25 | Open LLM Leaderboard | model author | 2026-10-07 |
| author-reported OpenAI’s LAMBADA | 0.720 ± 0.006 | Zero-shot | model author | 2026-10-07 |
| author-reported SciQ | 0.928 ± 0.008 | Zero-shot | model author | 2026-10-07 |
| author-reported TruthfulQA · Chat and follow instructions | 31.61 | Open LLM Leaderboard | model author | 2026-10-07 |
| author-reported Winogrande · Chat and follow instructions | 68.9 | Open LLM Leaderboard | model author | 2026-10-07 |
GPT-NeoX-20B is a 20.7B parameter autoregressive language model developed by EleutherAI for text generation.[1][2] The model uses a Transformer-based architecture and is released under the Apache 2.0 license.[3][4] It was trained on the Pile dataset using a sequence length of 2048.[5][6]
Generated 2026-10-07 15:05 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.
Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.
Listed in LLMs
| Pipeline tag | text-generation |
|---|---|
| Library | transformers |
| Architectures | GPTNeoXForCausalLM |
| Model type | gpt_neox |
| Dtype | float16 |
| Params by dtype | U8: 184,549,376 · F16: 20,554,568,208 |
| Size band | L |
| MoE | no |
| Context length | 2,048 (config.json max_position_embeddings) |
| Languages | en |
| Papers | arXiv:2204.06745, arXiv:2101.00027, arXiv:2201.07311, arXiv:2104.09864 |
| License | Apache 2.0 (apache-2.0) |
| Hub snapshot | 2026-10-25 |
vLLM: supported · GPTNeoXForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: not in registry · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed
vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.
Weights-only memory for gpt-neox-20b: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.
| BF16 (2 bytes/param) | 50 GB · Single high-end GPU (<80 GB) |
|---|---|
| FP8 (1 byte/param) | 25 GB · Single high-end GPU (<80 GB) |
| INT4 (0.5 bytes/param) | 12 GB · Single GPU (<24 GB) |
Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.
quantized: 9finetune: 9adapter: 25merge: 1
| Variant | Relation | Quant | Params | Band | License | Author | Likes | Downloads |
|---|---|---|---|---|---|---|---|---|
| mradermacher/gpt-neox-20b-i1-GGUF | quantized | GGUF | — | — | apache-2.0 | mradermacher | 0 | 29.9k |
| mradermacher/gpt-neox-20b-GGUF | quantized | GGUF | — | — | apache-2.0 | mradermacher | 1 | 12.1k |
| zhentaoyu/gpt-neox-20b-Q4_0-GGUF | quantized | Q4_0 | — | — | apache-2.0 | zhentaoyu | 0 | 9.7k |
| tensorblock/gpt-neox-20b-GGUF | quantized | GGUF | — | — | apache-2.0 | tensorblock | 1 | 5.6k |
| OpenVINO/gpt-neox-20b-int8-ov | quantized | int8 | — | — | apache-2.0 | OpenVINO | 1 | 431 |
| nhe-ai/gpt-neox-20b-mlx-4Bit | quantized | mlx | 20.6B | L | apache-2.0 | nhe-ai | 0 | 416 |
| OpenVINO/gpt-neox-20b-fp16-ov | finetune | -fp16 | — | — | apache-2.0 | OpenVINO | 0 | 339 |
| hipnologo/GPT-Neox-20b-QLoRA-FineTune-english_quotes_dataset | adapter | — | — | — | apache-2.0 | hipnologo | 0 | 268 |
| Mahler60/Prueba | finetune | — | — | — | — | Mahler60 | 0 | 246 |
| KalaiarasiS14/gpt-neox-20b-Q4_K_M-GGUF | quantized | GGUF | — | — | apache-2.0 | KalaiarasiS14 | 0 | 228 |
| KalaiarasiS14/gpt-neox-20b-Q4_0-GGUF | quantized | Q4_0 | — | — | apache-2.0 | KalaiarasiS14 | 0 | 154 |
| aidiary/gpt-neox-20b-finetuned-english-quotes | adapter | — | — | — | apache-2.0 | aidiary | 0 | 136 |
| Ogrex/Test-gpt-neox-20b | adapter | — | — | — | — | Ogrex | 0 | 114 |
| shuvom/gpt-neox-20B-FT_RD-service | adapter | — | — | — | mit | shuvom | 0 | 97 |
| Mohammed-Altaf/instruct-finetuned-medical-20b-adapters | adapter | — | — | — | mit | Mohammed-Altaf | 0 | 97 |
| CarlBrendt/gpt-neox-20b | adapter | — | — | — | — | CarlBrendt | 0 | 90 |
| 1bit-MONSTER/gpt-neox-20b-GGUF | quantized | GGUF | — | — | apache-2.0 | 1bit-MONSTER | 0 | 71 |
| motazel/llama-2-7b-chat-guanaco-motaz | adapter | — | — | — | — | motazel | 0 | 69 |
| nurke/PEFT-v1-alpaca | adapter | — | — | — | apache-2.0 | nurke | 0 | 67 |
| takumi123xxx/gpt-neox-20b-japanese-book-qlora | adapter | — | — | — | — | takumi123xxx | 0 | 49 |
| IAyamina/gptneo20b_on_instruction_poems | adapter | — | — | — | — | IAyamina | 0 | 43 |
| tank028/gpt-neox-20b-finet | adapter | — | — | — | apache-2.0 | tank028 | 0 | 42 |
| MinMaxolotl/outputs | adapter | — | — | — | apache-2.0 | MinMaxolotl | 0 | 39 |
| CarlBrendt/gpt-neox-20b_new | adapter | — | — | — | — | CarlBrendt | 0 | 37 |
| kzsnlsa/medbot-model | adapter | — | — | — | — | kzsnlsa | 0 | 36 |
| Electricarchmage/ApocryphaGenerator | adapter | — | — | — | mit | Electricarchmage | 0 | 35 |
| nadadb/medbot | adapter | — | — | — | — | nadadb | 0 | 35 |
| Abdelateef/medbot-lora | adapter | — | — | — | — | Abdelateef | 0 | 35 |
| asoria/EleutherAI-gpt-neox-20b-english_quotes | adapter | — | — | — | apache-2.0 | asoria | 0 | 31 |
| nqzfaizal77ai/ethereal-engine-x-reinit-618m-zero | finetune | — | 618M | S | — | nqzfaizal77ai | 0 | 30 |
| singhamal1710/cryptocompass | adapter | — | — | — | — | singhamal1710 | 0 | 26 |
| MinusV25/einsteinecho | adapter | — | — | — | — | MinusV25 | 0 | 24 |
| CarlBrendt/gpt-neox-20b-arif | adapter | — | — | — | — | CarlBrendt | 0 | 20 |
| FadilaOmar/Medbot-v2 | adapter | — | — | — | — | FadilaOmar | 0 | 20 |
| RudzC/medical_information | adapter | — | — | — | apache-2.0 | RudzC | 0 | 16 |
| Shyaboi/fine-tuned-adapters | adapter | — | — | — | — | Shyaboi | 0 | 2 |
| shivarama23/outputs | finetune | — | — | — | apache-2.0 | shivarama23 | 0 | 0 |
| solanotodeschini/outputs | finetune | — | — | — | apache-2.0 | solanotodeschini | 0 | 0 |
| paragon-analytics/outputs | finetune | — | — | — | apache-2.0 | paragon-analytics | 0 | 0 |
| vineetsharma/qlora-gpt-neox-20b-english_quotes | finetune | — | — | — | apache-2.0 | vineetsharma | 0 | 0 |
| Date | Commit | SHA |
|---|---|---|
| 2024-01-31 | Adding Evaluation Results (#25) | c292233 |
| 2023-06-21 | Update README.md | 9369f14 |
| 2023-02-09 | Create README.md (#14) | 4e49ead |
| 2023-02-07 | Set dropout in config.json to be 0 ? (#7) | 53bcd7e |
| 2023-02-07 | Adding `safetensors` variant of this model (#13) | 007b5cb |
| 2022-08-09 | Fix eos_token_id (#3) | 3523781 |
| 2022-04-07 | tokenizer | 364ae95 |
| 2022-04-07 | 20B first commit | 5fe656a |
| 2022-04-07 | initial commit | 2836058 |
Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.
arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).
Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:37 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:05 UTC · Last verified 2026-10-25
Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.
