Show HN: Llama 3.2 Interpretability with Sparse Autoencoders
The project trained a Sparse Autoencoder on 6.5 Billion activations from Llama 3.2-3B layer 23 over 10 epochs.
Up to date
meta-llama Llama 3.2 Community License out 2 years ago
3.21B parameters, text in, text out, 128K tokens of context. 3 providers serve it from $0.050 per million input tokens.
Verified 2026-10-25 Rank #105 in LLMs · score 98.7 method v1 Sources 3 Downloads 30d 228k · #239 of 619 tracked
Picked by rule from 36 tracked pieces: the lab's own post, the most shared, a field report, the latest.
The project trained a Sparse Autoencoder on 6.5 Billion activations from Llama 3.2-3B layer 23 over 10 epochs.
A minimal implementation of Llama 3.2 Instruct inference runs in ~100 lines of pure Numpy code.
The 3 picks above, plus these 22. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.
Live hosted endpoints, cheapest first, USD per million tokens. Ties break on output price, then provider name.
2 providers with a live price, from $0.050 to $0.051 per million input tokens; the median is $0.050. One dot per provider, cheapest in teal.
Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.
Served by 3 API providers; heavily run locally (156 quantized builds); a big base for builders (1,024 fine-tunes, adapters and merges).
Top Spaces running it: tomg-group-umd/lm-watermarking ♥ 100 · Steveeeeeeen/SpeechLLM-Playbook ♥ 42 · AiActivity/AI-Assistant ♥ 21 · artificialguybr/CanIRunThisLLM ♥ 9
Weights only, an estimate: leave room for the KV cache. Method
About 1.9 GB at INT4, 3.9 GB at FP8, 7.7 GB at BF16: weights only, so leave room for the KV cache.
Estimate params × bytes per param × 1.2 · how we estimate
Weights, every dated article, and repo commits on one rail.
Teal is the weights (2024-09-18), amber triangles are 10 dated articles (each links to it).
1,180 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.
From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.
Everything else, folded: the full tables, every source.
Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.
Listed in LLMs
| Pipeline tag | text-generation |
|---|---|
| Library | transformers |
| Architectures | LlamaForCausalLM |
| Model type | llama |
| Params by dtype | BF16: 3,212,749,824 |
| Size band | M |
| MoE | no |
| Context length | 131,072 (provider listing) |
| Gated | manual |
| Papers | arXiv:2204.05149, arXiv:2405.16406 |
| License | Llama 3.2 Community License (llama3.2) |
| Hub snapshot | 2026-10-25 |
vLLM: supported · LlamaForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · LlamaForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed
vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.
Weights-only memory for Llama-3.2-3B: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.
| BF16 (2 bytes/param) | 7.7 GB · Single GPU (<24 GB) |
|---|---|
| FP8 (1 byte/param) | 3.9 GB · Edge (<4 GB) |
| INT4 (0.5 bytes/param) | 1.9 GB · Edge (<4 GB) |
Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.
finetune: 547quantized: 156adapter: 441merge: 36
| Variant | Relation | Quant | Params | Band | License | Author | Likes | Downloads |
|---|---|---|---|---|---|---|---|---|
| unsloth/Llama-3.2-3B | finetune | — | 3.21B | M | llama3.2 | unsloth | 18 | 732.9k |
| unsloth/Llama-3.2-3B-bnb-4bit | quantized | -bnb | 3.30B | M | llama3.2 | unsloth | 23 | 502.8k |
| HumeAI/tada-3b-ml | finetune | — | 4.23B | M | llama3.2 | HumeAI | 161 | 258.4k |
| meta-llama/Llama-3.2-3B-Instruct-SpinQuant_INT4_EO8 | finetune | — | — | — | llama3.2 | meta-llama | 40 | 116.3k |
| lemuralabs/Llama-3.2-3B-Uncensored-GGUF | adapter | GGUF | — | — | apache-2.0 | lemuralabs | 53 | 112.4k |
| dphn/Dolphin3.0-Llama3.2-3B | finetune | — | 3.21B | M | llama3.2 | dphn | 76 | 64.5k |
| QuantFactory/OneLLM-Doey-V1-Llama-3.2-3B-GGUF | quantized | GGUF | — | — | apache-2.0 | QuantFactory | 2 | 35.2k |
| deepcogito/cogito-v1-preview-llama-3B | finetune | — | 3.61B | M | llama3.2 | deepcogito | 102 | 28.7k |
| MaziyarPanahi/calme-3.1-llamaloi-3b | finetune | — | 3.21B | M | llama3.2 | MaziyarPanahi | 1 | 33.7k |
| ABDALLALSWAITI/llm_3_2_flux_prompt | quantized | GGUF | 3.21B | M | llama3.2 | ABDALLALSWAITI | 15 | 30.8k |
| nvidia/llama-nv-embed-reasoning-3b | finetune | — | 3.21B | M | cc-by-4.0 | nvidia | 28 | 29.3k |
| lastmass/llama3.2-chinese | quantized | GGUF | — | — | apache-2.0 | lastmass | 7 | 24k |
| Mungert/Dolphin3.0-Llama3.2-3B-GGUF | quantized | GGUF | — | — | llama3.2 | Mungert | 4 | 22.3k |
| OctoThinker/OctoThinker-3B-Long-Base | finetune | — | 3.21B | M | llama3.2 | OctoThinker | 1 | 21.8k |
| OctoThinker/OctoThinker-3B-Hybrid-Base | finetune | — | 3.21B | M | llama3.2 | OctoThinker | 1 | 21.3k |
| OctoThinker/OctoThinker-8B-Hybrid-Base | finetune | — | 8.03B | M | llama3.2 | OctoThinker | 2 | 17.3k |
| pangram/editlens_Llama-3.2-3B | adapter | — | — | — | cc-by-nc-sa-4.0 | pangram | 36 | 15.4k |
| meta-llama/Llama-3.2-3B-Instruct-QLORA_INT4_EO8 | finetune | — | — | — | llama3.2 | meta-llama | 76 | 11k |
| LTS-VVE/Teuta | quantized | GGUF | — | — | apache-2.0 | LTS-VVE | 3 | 10.7k |
| twinkle-ai/Llama-3.2-3B-F1-Instruct | finetune | — | 3.61B | M | llama3.2 | twinkle-ai | 27 | 6.9k |
| QuantFactory/Dolphin3.0-Llama3.2-3B-GGUF | quantized | GGUF | — | — | llama3.2 | QuantFactory | 3 | 7.8k |
| Shyamnath/Llama-3.2-3b-Uncensored-GGUF | quantized | GGUF | — | — | apache-2.0 | Shyamnath | 6 | 7k |
| QuantFactory/Llama-3.2-Taiwan-3B-GGUF | quantized | GGUF | — | — | llama3.2 | QuantFactory | 4 | 6.6k |
| slprl/SIMS-Llama3.2-3B | finetune | — | 3.21B | M | llama3.2 | slprl | 2 | 6.7k |
| bodhan-ai/indic-speak | finetune | — | 3.30B | M | other | bodhan-ai | 46 | 4.5k |
| NeuraLakeAi/iSA-03-Mini-3B-Hybrid-Preview | finetune | — | — | — | apache-2.0 | NeuraLakeAi | 4 | 6.5k |
| OctoThinker/OctoThinker-3B-Short-Base | finetune | — | 3.21B | M | llama3.2 | OctoThinker | 0 | 6.4k |
| michelkao/Ollama-3.2-GGUF | quantized | GGUF | — | — | creativeml-openrail-m | michelkao | 0 | 5.8k |
| prithivMLmods/Llama-3.2-3B-GGUF | quantized | GGUF | — | — | creativeml-openrail-m | prithivMLmods | 2 | 5.4k |
| rshwndsz/Llama-3.2-3B-SFT | finetune | — | 3.21B | M | — | rshwndsz | 1 | 5.3k |
| lianghsun/Llama-3.2-Taiwan-3B | finetune | — | 3.61B | M | llama3.2 | lianghsun | 31 | 3.6k |
| OctoThinker/OctoThinker-8B-Short-Base | finetune | — | 8.03B | M | llama3.2 | OctoThinker | 1 | 4.9k |
| HuggingFaceTB/FineMath-Llama-3B | finetune | — | 3.21B | M | apache-2.0 | HuggingFaceTB | 23 | 3.2k |
| brianknowsai/Brian-Llama-3.2-3B | finetune | — | 3.21B | M | llama3.2 | brianknowsai | 1 | 4k |
| hishab/titulm-llama-3.2-3b-v1.1 | finetune | — | 3.21B | M | llama3.2 | hishab | 0 | 3.6k |
| activeDap/Llama-3.2-3B_hh_harmful | finetune | — | 3.21B | M | apache-2.0 | activeDap | 1 | 3.4k |
| sai-santhosh/text-2-sql-Llama-3.2-3B | finetune | — | 3.21B | M | apache-2.0 | sai-santhosh | 0 | 3.4k |
| hathibelagal/llama-3.2-latin | finetune | — | 3.21B | M | — | hathibelagal | 2 | 3.3k |
| ericflo/Llama-3.2-3B-COT | quantized | GGUF | 3.21B | M | apache-2.0 | ericflo | 2 | 2.7k |
| Gurubot/self-after-dark-3b | quantized | GGUF | 3.61B | M | apache-2.0 | Gurubot | 3 | 2.6k |
Hosted endpoints observed for Llama-3.2-3B, sorted by input price (USD per million tokens). Each row names its observer and date; prices change daily, so check the source before quoting. Cheapest observed input price: $0.05/M.
| Provider | Model ref | Context | Quant | $/M in | $/M out | tok/s | TTFT | Tools | Status | Observed |
|---|---|---|---|---|---|---|---|---|---|---|
| Parasail openrouter | meta-llama/llama-3.2-3b-instruct | 128K | bf16 | $0.05 | $0.33 | — | — | — | live | 2026-10-24 |
| Cloudflare openrouter | meta-llama/llama-3.2-3b-instruct | 78K | unknown | $0.05 | $0.34 | — | — | — | live | 2026-10-24 |
| Featherless hf | meta-llama/Llama-3.2-3B | — | — | — | — | — | — | — | live | 2026-10-25 |
Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.
arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).
Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:34 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:01 UTC · Last verified 2026-10-25
Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.