Show HN: Llama 3.2 Interpretability with Sparse Autoencoders
The project trained a Sparse Autoencoder on 6.5 Billion activations from Llama 3.2-3B layer 23 over 10 epochs.
Up to date
meta-llama Llama 3.2 Community License out 2 years ago
10.6B parameters, images and text in, text out.
Verified 2026-10-25 Rank #516 in LLMs · score 75.9 method v1 Sources 3 Downloads 30d 12.2k · #494 of 619 tracked
Picked by rule from 25 tracked pieces: the lab's own post, the most shared, a field report, the latest.
The project trained a Sparse Autoencoder on 6.5 Billion activations from Llama 3.2-3B layer 23 over 10 epochs.
A minimal implementation of Llama 3.2 Instruct inference runs in ~100 lines of pure Numpy code.
The 2 picks above, plus these 23. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.
Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.
No API provider serves it, so its downloads are people running it themselves: 3 quantized builds and 75 Spaces.
Top Spaces running it: TIGER-Lab/MEGA-Bench ♥ 51 · Aashi/Text-Image-Analyzer ♥ 18 · guangyangmusic/legato-demo ♥ 8 · sailfish/explain-image ♥ 2
Weights only, an estimate: leave room for the KV cache. Method
About 6.4 GB at INT4, 13 GB at FP8, 26 GB at BF16: weights only, so leave room for the KV cache.
Estimate params × bytes per param × 1.2 · how we estimate
Weights, every dated article, and repo commits on one rail.
Teal is the weights (2024-09-18), amber triangles are 10 dated articles (each links to it).
23 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.
From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.
Everything else, folded: the full tables, every source.
Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.
Listed in LLMs
| Pipeline tag | image-text-to-text |
|---|---|
| Library | transformers |
| Architectures | MllamaForConditionalGeneration |
| Model type | mllama |
| Params by dtype | BF16: 10,642,941,475 |
| Size band | L |
| MoE | no |
| Gated | manual |
| Papers | arXiv:2204.05149 |
| License | Llama 3.2 Community License (llama3.2) |
| Hub snapshot | 2026-10-25 |
vLLM: not in registry · MllamaForConditionalGeneration · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · MllamaForConditionalGeneration · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed
vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.
Weights-only memory for Llama-3.2-11B-Vision: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.
| BF16 (2 bytes/param) | 26 GB · Single high-end GPU (<80 GB) |
|---|---|
| FP8 (1 byte/param) | 13 GB · Single GPU (<24 GB) |
| INT4 (0.5 bytes/param) | 6.4 GB · Single GPU (<24 GB) |
Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.
finetune: 9quantized: 3adapter: 11
| Variant | Relation | Quant | Params | Band | License | Author | Likes | Downloads |
|---|---|---|---|---|---|---|---|---|
| unsloth/Llama-3.2-11B-Vision | finetune | — | 10.6B | L | llama3.2 | unsloth | 34 | 27.6k |
| unsloth/Llama-3.2-11B-Vision-bnb-4bit | quantized | -bnb | 10.9B | L | llama3.2 | unsloth | 19 | 20.4k |
| unsloth/Llama-3.2-11B-Vision-unsloth-bnb-4bit | quantized | -bnb | 10.9B | L | llama3.2 | unsloth | 5 | 17.3k |
| impactframes/Llama-3.2-11B-Vision-bnb-4bit | quantized | -bnb | 10.9B | L | llama3.2 | impactframes | 0 | 1.6k |
| shashikanth-a/Llama-3.2-11B-Vision-4bit | finetune | -4bit | 7.06B | M | llama3.2 | shashikanth-a | 2 | 1.3k |
| PKU-Alignment/Beaver-Vision-11B | finetune | — | — | — | apache-2.0 | PKU-Alignment | 2 | 939 |
| Infernaught/mllama-3.2-11b-10-epochs | adapter | — | — | — | — | Infernaught | 1 | 65 |
| Mahesh-janu/vision-textile | finetune | — | 124M | S | llama3.2 | Mahesh-janu | 0 | 107 |
| avi1057/lora | adapter | — | — | — | llama3.2 | avi1057 | 0 | 70 |
| yilin-wu/lora | adapter | — | — | — | llama3.2 | yilin-wu | 0 | 62 |
| aruboi/llama-32-11b-vision_model_output22 | finetune | — | — | — | llama3.2 | aruboi | 0 | 62 |
| masud99r/lora | adapter | — | — | — | llama3.2 | masud99r | 0 | 53 |
| mgertner/lora | adapter | — | — | — | llama3.2 | mgertner | 0 | 47 |
| kvu0/VQA | adapter | — | — | — | — | kvu0 | 0 | 40 |
| aruboi/llama-32-11b-vlm_peft_output33 | adapter | — | — | — | llama3.2 | aruboi | 0 | 38 |
| tangy0/lora | adapter | — | — | — | llama3.2 | tangy0 | 0 | 26 |
| a22aniket/paligemma_vqav2 | adapter | — | — | — | llama3.2 | a22aniket | 0 | 24 |
| mgertner/llama-collegiate2 | adapter | — | — | — | — | mgertner | 0 | 24 |
| Infernaught/mllama-11b-vision-lora | adapter | — | — | — | — | Infernaught | 0 | 23 |
| igoruehara/llama | finetune | — | — | — | mit | igoruehara | 0 | 0 |
| CHENG678/Llama3.2_11B | finetune | — | — | — | llama3.2 | CHENG678 | 0 | 0 |
| harshg9/Llama-3.2-Test-A | finetune | — | — | — | llama3.2 | harshg9 | 0 | 0 |
| Adignite/Llama-3.2-11B-Vision-image_tuned | finetune | — | — | — | mit | Adignite | 0 | 0 |
Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.
arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).
Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:39 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:46 UTC · Summary: 2026-10-07 15:10 UTC · Last verified 2026-10-25
Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.