Model Follower.

Up to date

Rebuilt
2026-10-25 06:54 UTC
Release radar
117 on watch
Hub snapshot
2026-10-25
Re-checked
daily · every number sourced and dated

Models › distilgpt2 family

distilgpt2

Hugging Face ↗ Follow changes

distilbert Apache 2.0 out 5 years ago

88.2M parameters, text in, text out, 1K tokens of context.

Verified 2026-10-25 Rank #181 in LLMs · score 90.3 method v1 Sources 3 Downloads 30d 1.7M · #75 of 619 tracked

Can I run it?0.1GB at INT4Estimatefits a 16 GB cardmethodCommunity: distilgpt2: parameters, size and GPU requirements - Model Anatomy →
Can I use it?Apache 2.0Permissive; patent grantlicence file2026-10-24
Do people use it?1.7Mdownloads, 30 days#75 of 619 we track · top 12%+5 likes, downloads −7% this week661 likesHugging Face2026-10-24

Worth reading

Picked by rule from 5 tracked pieces: the lab's own post, the most shared, a field report, the latest.

5 best of 5 tracked

The 2 picks above, plus these 3. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.

  1. Articledistilgpt2: parameters, size and GPU requirements - Model Anatomymodelanatomy.com · undated
  2. ArticleDistilGPT2 by distilbert | The GTM Directorythegtmdirectory.com · undated
  3. ArticleDeploy a Text Generating API with Hugging Face’s DistilGPT-2towardsdatascience.com · 2019-11-18 · 1 HN pts

Where it's used

Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.

No API provider serves it, so its downloads are people running it themselves: 24 quantized builds and 100 Spaces.

  • As a service0no API provider listed
  • On own hardware24quantized builds
  • Built on1,6661,578 fine-tunes · 86 adapters · 2 merges
  • In apps100+Hugging Face Spaces

Top Spaces running it: pliny-the-prompter/obliteratus ♥ 621 · h2oai/h2ogpt-chatbot ♥ 295 · exbert-project/exbert ♥ 187 · embedl/hfviewer ♥ 181

Will it fit?

Weights only, an estimate: leave room for the KV cache. Method

Fits a 16 GB card at INT4

About 0.1 GB at INT4, 0.1 GB at FP8, 0.2 GB at BF16: weights only, so leave room for the KV cache.

16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 0.1 GB of weights (estimate; KV cache and activations extra)INT40.1 GBFP8: about 0.1 GB of weights (estimate; KV cache and activations extra)FP80.1 GBBF16: about 0.2 GB of weights (estimate; KV cache and activations extra)BF160.2 GB
16 GB card64 GB Mac2×80 GB8×80 GBINT4: about 0.1 GB of weights (estimate; KV cache and activations extra)INT40.1 GBFP8: about 0.1 GB of weights (estimate; KV cache and activations extra)FP80.1 GBBF16: about 0.2 GB of weights (estimate; KV cache and activations extra)BF160.2 GB

Estimate params × bytes per param × 1.2 · how we estimate

The story so far

Weights, every dated article, and repo commits on one rail.

2 dated articles and 30 repo commits so far

Teal is the weights (2022-03-02), amber triangles are 2 dated articles (each links to it), grey ticks are repo commits.

NovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2024-02-19: 1 repo commit2023-04-29: 1 repo commit2023-01-24: 4 repo commits2022-11-14: 1 repo commit2022-09-05: 1 repo commit2022-07-22: 1 repo commit2022-06-02: 1 repo commit2022-05-26: 1 repo commit2021-05-21: 3 repo commits2020-12-11: 1 repo commit2020-12-09: 1 repo commit2020-10-12: 1 repo commit2020-05-11: 1 repo commit2020-04-24: 2 repo commits2020-01-31: 1 repo commit2019-12-20: 1 repo commit2019-11-29: 1 repo commit2019-10-07: 2 repo commits2019-10-03: 5 repo commitsSwift Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT · 2022-12-04Deploy a Text Generating API with Hugging Face’s DistilGPT-2 · 2019-11-18Weights 2022-03-02 · 2022-03-02Weights 2022-03-02now
NovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJulAugSepOct2024-02-19: 1 repo commit2023-04-29: 1 repo commit2023-01-24: 4 repo commits2022-11-14: 1 repo commit2022-09-05: 1 repo commit2022-07-22: 1 repo commit2022-06-02: 1 repo commit2022-05-26: 1 repo commit2021-05-21: 3 repo commits2020-12-11: 1 repo commit2020-12-09: 1 repo commit2020-10-12: 1 repo commit2020-05-11: 1 repo commit2020-04-24: 2 repo commits2020-01-31: 1 repo commit2019-12-20: 1 repo commit2019-11-29: 1 repo commit2019-10-07: 2 repo commits2019-10-03: 5 repo commitsSwift Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT · 2022-12-04Deploy a Text Generating API with Hugging Face’s DistilGPT-2 · 2019-11-18Weights 2022-03-02 · 2022-03-02Weights 2022-03-02now

What people built on it

1,690 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.

1,690 derivatives on the Hub, most of them fine-tunes

From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.

24 quantized (Hugging Face model tree)1,578 fine-tunes (Hugging Face model tree)2 merges (Hugging Face model tree)86 adapters (Hugging Face model tree)24 quantized1,578 fine-tunes2 merges86 adapters
24 quantized (Hugging Face model tree)1,578 fine-tunes (Hugging Face model tree)2 merges (Hugging Face model tree)86 adapters (Hugging Face model tree)24 quantized1,578 fine-tunes2 merges86 adapters
GGUF QuantFactory/distilgpt2-finetuned-databricks-GGUF for llama.cpp, Ollama, LM Studio 6.4k downloads

All variants →

Reference

Everything else, folded: the full tables, every source.

What the card says

1 cited sentence

Summary generated, cited

DistilGPT2 is an English-language transformer-based language model designed for text generation.[1][2]

  1. “Transformer-based Language Model”
  2. pipeline tag

Generated 2026-10-07 15:04 UTC by parasail-gemma-4-26b-a4b-it; each numeral points at the fact it rests on. Not a verdict.

Use-case evidence

1 use case

What it is good at 1 use case

Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.

Facts

20 fields

Listed in LLMs

Facts

Pipeline tagtext-generation
Librarytransformers
ArchitecturesGPT2LMHeadModel
Model typegpt2
Params by dtypeF32: 88,204,032
Size bandS
MoEno
Context length1,024 (config.json max_position_embeddings)
Languagesen
PapersarXiv:1910.01108, arXiv:2201.08542, arXiv:2203.12574, arXiv:1910.09700, arXiv:1503.02531
LicenseApache 2.0 (apache-2.0)
Hub snapshot2026-10-25

Servability

vLLM · SGLang

Servability

vLLM: supported · GPT2LMHeadModel · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · GPT2LMHeadModel · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed

vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.

Hardware fit

3 estimates

Hardware fit ESTIMATE

Weights-only memory for distilgpt2: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.

BF16 (2 bytes/param)0.2 GB · Edge (<4 GB)
FP8 (1 byte/param)0.1 GB · Edge (<4 GB)
INT4 (0.5 bytes/param)0.1 GB · Edge (<4 GB)

Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.

All variants

40 variants

Lineage

Original model Family: distilbert/distilgpt2 1,690 derivatives on the Hub

quantized: 24finetune: 1,578adapter: 86merge: 2

Top variants (40 shown, ranked by downloads)

Relation
40 rows
Variant Relation Quant Params Band License Author Likes Downloads
Xenova/distilgpt2 quantized — — — — Xenova 13 181.2k
as-cle-bert/distilgpt2-finetuned-microbiology finetune — 81.9M S apache-2.0 as-cle-bert 2 61.8k
postbot/distilgpt2-emailgen finetune — 88.2M S apache-2.0 postbot 4 14.2k
QuantFactory/distilgpt2-finetuned-databricks-GGUF quantized GGUF — — apache-2.0 QuantFactory 1 6.4k
mradermacher/distilgpt2-i1-GGUF quantized GGUF — — apache-2.0 mradermacher 0 5.8k
QuantFactory/distilgpt2-finetuned-python_code_instructions_18k_alpaca-GGUF quantized GGUF — — apache-2.0 QuantFactory 2 5.6k
ANOUARELIDRISSI/email-generator-gpt2 finetune — 81.9M S — ANOUARELIDRISSI 0 4.5k
itsmechico/distilgpt2 finetune — 88.2M S apache-2.0 itsmechico 0 4.2k
tensorblock/distilgpt2-GGUF quantized GGUF — — apache-2.0 tensorblock 0 4k
mariamoracrossitcr/distilgpt2_finetuneWithEli5 finetune — — — apache-2.0 mariamoracrossitcr 0 3.7k
rwang5688/distilgpt2-finetuned-wikitext2-pt finetune — 81.9M S apache-2.0 rwang5688 1 3.3k
ehcalabres/distilgpt2-abc-irish-music-generation finetune — 88.2M S apache-2.0 ehcalabres 10 2.8k
mradermacher/distilgpt2-GGUF quantized GGUF — — apache-2.0 mradermacher 0 2.8k
0Tick/e621TagAutocomplete finetune — 81.9M S mit 0Tick 5 2.4k
serene89104/distilgpt2-finetuned-wikitext2 finetune — 81.9M S apache-2.0 serene89104 0 2.5k
microsoft/chatbench-distilgpt2 finetune — 81.9M S apache-2.0 microsoft 7 1.9k
0Tick/danbooruTagAutocomplete finetune — 81.9M S mit 0Tick 1 1.9k
wren11ws/sunup finetune — 81.9M S apache-2.0 wren11ws 1 1.8k
nebiyuelias1/distilgpt2-finetuned-cybernative finetune — 81.9M S apache-2.0 nebiyuelias1 0 1.8k
gweltou/tiny-gpt2-br finetune — 81.9M S apache-2.0 gweltou 0 1.7k
mafgit/empathetic-distilgpt2 finetune — 81.9M S apache-2.0 mafgit 0 1.7k
lottienghiem/distilgpt2-finetuned-wikitext2 finetune — 81.9M S apache-2.0 lottienghiem 0 1.5k
FlameF0X/MathGPT2 finetune — 81.9M S mit FlameF0X 3 1.2k
rwang5688/distilgpt2-finetuned-wikitext2 finetune — 81.9M S apache-2.0 rwang5688 1 1.3k
bkwalsh/distilgpt2-finetuned-wikitext2 finetune — 81.9M S apache-2.0 bkwalsh 0 1.3k
occasion-2/my_awesome_eli5_clm-model finetune — 81.9M S apache-2.0 occasion-2 0 1.3k
Nirwa22/Fine_tuned_model_final finetune — 81.9M S apache-2.0 Nirwa22 0 1.3k
Vyk223/model_output finetune — 81.9M S apache-2.0 Vyk223 0 1.2k
echarlaix/distilgpt2-openvino finetune — — — apache-2.0 echarlaix 0 1.2k
chinhon/distilgpt2-sgnews finetune — — — apache-2.0 chinhon 0 1.2k
isinghhug/distilgpt2-finetuned-wikitext2 finetune — 81.9M S apache-2.0 isinghhug 0 1.2k
shcapybara0925/distilgpt2-finetuned-github_cybersecurity_READMEs finetune — 81.9M S apache-2.0 shcapybara0925 0 1.1k
bestofbothworldsenjoyer/gpt2-expanded-test-distilled finetune — 81.9M S apache-2.0 bestofbothworldsenjoyer 0 1.1k
tarekziade/deit-tiny-distilgpt2 quantized — 102M S apache-2.0 tarekziade 2 974
tarekziade/distilvit quantized — 182M S apache-2.0 tarekziade 2 951
we0rr9u89q/chess_gpt2_results finetune — 81.9M S apache-2.0 we0rr9u89q 0 1k
aixin2024/codeparrot-ds finetune — 81.9M S apache-2.0 aixin2024 0 1k
sickcell/distilgpt2-finetuned-github_cybersecurity_READMEs finetune — 81.9M S apache-2.0 sickcell 0 1k
Alirani/synopsis_generator finetune — — — apache-2.0 Alirani 0 975
Passpass119/videogame-title-generator finetune — 81.9M S apache-2.0 Passpass119 2 866

Repository changelog

30 commits

Repository changelog 30 commits

DateCommitSHA
2024-02-19 Adds the tokenizer configuration file (#11) 2290a62
2023-04-29 Add Core ML conversion (#10) 38cc92e
2023-01-24 Adding generation config file(s) f241065
2023-01-24 Delete generation_config.json a748e05
2023-01-24 Delete {pipeline}_generation_config.json 32007bf
2023-01-24 Adding generation config file(s) 9e4cf98
2022-11-14 Adding `safetensors` variant of this model (#7) ee0b29d
2022-09-05 Add CoreML model (#6) c3772e6
2022-07-22 Fix emissions metadata (#5) ca98be8
2022-06-02 Limit use of collapsible sections; fix emissions info (#3) b754fe8
2022-05-26 New model card for distilgpt2 (#1) 737e343
2021-05-21 upload flax model 96699c9
2021-05-21 allow flax 2ddd647
2021-05-21 allow flax 92838a2
2020-12-11 Migrate model card from transformers-repo 9851938
2020-12-09 For clarity, delete deprecated modelcard.json 58ada35
2020-10-12 Update tokenizer.json b6d23df
2020-05-11 Update config.json e87a92b
2020-04-24 Update rust_model.ot 17ef8ca
2020-04-24 Update config.json b09826e
2020-01-31 Update config.json 51ab088
2019-12-20 Update modelcard.json baa9aea
2019-11-29 Update 64.tflite 2677794
2019-10-07 Update pytorch_model.bin 36840f1
2019-10-07 Update tf_model.h5 b70d85d
2019-10-03 Update config.json 0799984
2019-10-03 Update merges.txt 0c7d1e1
2019-10-03 Update config.json 09daaa0
2019-10-03 Update vocab.json db73fa3
2019-10-03 initial commit 1b04b2c

Articles and coverage

5 pieces

Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.

Analysis5

Swift Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT
github.com2022-12-04authorityHN 1 pts · 0 comments228 words
The archived swift-coreml-transformers repo offers Swift Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT models fine-tuned for question answering on the SQuAD dataset. For an in-development replacement, developers should head over to swift-transformers. The BERTSQUADFP16 Core ML model was demoed at WWDC 2019, and large model files require git-lfs to obtain necessary files for the demo apps.written, reviewed, cited
distilgpt2: parameters, size and GPU requirements - Model Anatomy
modelanatomy.com480 words
distilgpt2 is a dense transformer model with 88.2M parameters that requires 353 MB of disk space. The model supports a maximum context of 1K tokens and runs on a single 8 GB card like the RTX 4060. It was created on Mar 2, 2022 and is available under the apache-2.0 license.written, reviewed, cited
Distilgpt2 By distilbert: Benchmarks, Features and Detailed Analysis. Insights on Distilgpt2.
llm-explorer.com
DistilGPT2 by distilbert | The GTM Directory
thegtmdirectory.com1,290 words
DistilGPT2 is an 82-million-parameter, knowledge-distilled compression of GPT-2 released under the Apache 2.0 license on 2022-03-02. Builders can wire it into a Hugging Face Transformers pipeline with minimal setup, making it a low-friction starting point for offline edge deployments where a GPU is unavailable. At 82 million parameters, it runs on CPU with low memory overhead, but its perplexity of 21.1 on WikiText-103 versus GPT-2's 16.3 reflects meaningfully weaker coherence, and its English-only constraint limits it to non-critical generation tasks.written, reviewed, cited
  • DistilGPT2 is a throwaway demo chatbot option for hackathons rather than customer-facing outreach drafting.
  • DeepSeek V4 Pro activates only 49B parameters per forward pass despite having 1.6T total parameters.
  • Kimi K2.6 is an open-weight frontier chat model built for design-driven code generation and agentic task decomposition.
Deploy a Text Generating API with Hugging Face’s DistilGPT-2
towardsdatascience.com2019-11-18HN 1 pts · 0 comments
The provided webpage displays a 'Page not found' error instead of the intended article about deploying a text-generating API. The page content consists entirely of a cookie consent banner listing various tracking cookies from services like Cloudflare and YouTube.generated, cited

Papers

5 papers

Papers 5

arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).

Similar models

3 models

Similar models

  • pythia-70m-deduped · 95.6M
    same size band (S), same task (text-generation), permissive license
  • gpt2 · 137M
    same size band (S), same task (text-generation), permissive license
  • openai-gpt · 120M
    same size band (S), same task (text-generation), permissive license

Sources

3 sources

Sources

Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:36 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:04 UTC · Last verified 2026-10-25

Follow distilgpt2

Watch this model

Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.