Up to date
nvidia Llama 3.1 Community License out 2 years ago
70.6B parameters, text in, text out, 128K tokens of context. 1 provider serves it.
Verified 2026-10-25 Rank #263 in LLMs · score 85.3 method v1 Sources 3 Downloads 30d 23.1k · #444 of 619 tracked
Picked by rule from 33 tracked pieces: the lab's own post, the most shared, a field report, the latest.
g1 is an experimental prototype using prompting strategies to improve Llama-3.1-70b reasoning capabilities through o1-like chains.
ntransformer is a C++/CUDA inference engine that runs Llama 70B on a single RTX 3090 using a 3-tier adaptive caching system with VRAM-resident layers, pinned RAM, and NVMe/mmap fallback.
The 3 picks above, plus these 22. Top is the radar's score: discussion, source authority and the lab's own posts, fading with age.
Live hosted endpoints, cheapest first, USD per million tokens. Ties break on output price, then provider name.
| # | Cheapest live row | In /1M | Out /1M | Context | Quant | Seen |
|---|---|---|---|---|---|---|
| 1 | Featherless | — | — | — | — | Oct 25 |
Hugging Face does not say who downloads a model or with what. These are the proxies: API providers listing it, quantized builds, fine-tunes/adapters/merges, and Spaces that load it.
Served by 1 API provider; 45 quantized builds for local use; a big base for builders (119 fine-tunes, adapters and merges).
Top Spaces running it: pliny-the-prompter/obliteratus ♥ 621 · AiActivity/AI-Assistant ♥ 21 · ZongqianLi/ReasonGraph ♥ 14 · hacktogone/uncensored-chat ♥ 6
Weights only, an estimate: leave room for the KV cache. Method
About 42 GB at INT4, 85 GB at FP8, 169 GB at BF16: weights only, so leave room for the KV cache.
Estimate params × bytes per param × 1.2 · how we estimate
Weights, every dated article, and repo commits on one rail.
Teal is the weights (2024-10-12), amber triangles are 9 dated articles (each links to it), grey ticks are repo commits.
164 derivatives on the Hub. Which build to run, by runtime: the most downloaded in each class.
From the Hugging Face model tree; quantized builds in teal, fine-tunes, merges and adapters in greys.
Everything else, folded: the full tables, every source.
| Benchmark | Value | Setting / notes | Reported by | Date |
|---|---|---|---|---|
| author-reported Arena-Hard · Chat and follow instructions | 85 | — | model author | 2026-10-07 |
| author-reported MT-Bench · Chat and follow instructions | 8.98 | (GPT-4-Turbo) | model author | 2026-10-07 |
Labels are evidence, not verdicts: measured a third-party leaderboard row for this model; author-reported a number the model author published in the card or eval files; structural a deterministic signal from the name, config or chat template, or a decision-model estimate. Each chip links to its source. All use cases.
Listed in LLMs
| Pipeline tag | text-generation |
|---|---|
| Library | transformers |
| Architectures | LlamaForCausalLM |
| Model type | llama |
| Dtype | bfloat16 |
| Params by dtype | BF16: 70,553,706,496 |
| Size band | XL |
| MoE | no |
| Context length | 131,072 (config.json max_position_embeddings) |
| Languages | en |
| Papers | arXiv:2410.01257, arXiv:2405.01481, arXiv:2406.08673 |
| License | Llama 3.1 Community License (llama3.1) |
| Hub snapshot | 2026-10-25 |
vLLM: supported · LlamaForCausalLM · 7050ef881ff0835f6890a47db1859a24c65481d6SGLang: supported · LlamaForCausalLM · 773cb938fe9484a8f7e7e46fb1b1fe96412c59ed
vLLM registry (7050ef881ff0835f6890a47db1859a24c65481d6) · SGLang registry (773cb938fe9484a8f7e7e46fb1b1fe96412c59ed) · Matched by architecture class name against each project's registry on main; 'supported' means the architecture is registered, not that this checkpoint was tested.
Weights-only memory for Llama-3.1-Nemotron-70B-Instruct-HF: params × bytes/param × 1.2 overhead. KV cache and activations are extra. How this is computed.
| BF16 (2 bytes/param) | 169 GB · Multi-GPU node (<640 GB) |
|---|---|
| FP8 (1 byte/param) | 85 GB · Multi-GPU node (<640 GB) |
| INT4 (0.5 bytes/param) | 42 GB · Single high-end GPU (<80 GB) |
Weights only (params × bytes/param × 1.2 overhead); KV cache and activations extra.
Base model: meta-llama/Llama-3.1-70B-Instruct
quantized: 45finetune: 57merge: 39adapter: 23
| Variant | Relation | Quant | Params | Band | License | Author | Likes | Downloads |
|---|---|---|---|---|---|---|---|---|
| mlx-community/nvidia_Llama-3.1-Nemotron-70B-Instruct-HF_4bit | quantized | — | 70.6B | XL | llama3.1 | mlx-community | 12 | 807.2k |
| mlx-community/Llama-3.1-Nemotron-70B-Instruct-HF-bf16 | finetune | -bf16 | 70.6B | XL | llama3.1 | mlx-community | 1 | 683.5k |
| bartowski/Llama-3.1-Nemotron-70B-Instruct-HF-GGUF | quantized | GGUF | — | — | llama3.1 | bartowski | 111 | 332.4k |
| RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-dynamic | quantized | -FP8 | 70.6B | XL | llama3.1 | RedHatAI | 15 | 173.5k |
| lmstudio-community/Llama-3.1-Nemotron-70B-Instruct-HF-GGUF | quantized | GGUF | — | — | llama3.1 | lmstudio-community | 39 | 49.8k |
| unsloth/Llama-3.1-Nemotron-70B-Instruct-bnb-4bit | quantized | -bnb | 72.7B | XL | llama3.1 | unsloth | 20 | 41.4k |
| nbeerbower/Llama-3.1-Nemotron-lorablated-70B | merge | — | 70.6B | XL | llama3.1 | nbeerbower | 16 | 33.9k |
| mradermacher/Llama-3.1-Nemotron-70B-Instruct-HF-i1-GGUF | quantized | GGUF | — | — | llama3.1 | mradermacher | 6 | 25.2k |
| ibnzterrell/Nvidia-Llama-3.1-Nemotron-70B-Instruct-HF-AWQ-INT4 | quantized | AWQ | 70.6B | XL | llama3.1 | ibnzterrell | 6 | 22.9k |
| joshmiller656/Llama-3.1-Nemotron-70B-Instruct-AWQ-INT4 | quantized | AWQ | 70.6B | XL | llama3.1 | joshmiller656 | 3 | 15.6k |
| mradermacher/Llama-3.1-Nemotron-70B-Instruct-HF-GGUF | quantized | GGUF | — | — | llama3.1 | mradermacher | 4 | 13.1k |
| second-state/Llama-3.1-Nemotron-70B-Instruct-HF-GGUF | quantized | GGUF | — | — | llama3.1 | second-state | 0 | 10.7k |
| mlx-community/Llama-3.1-Nemotron-70B-Instruct-HF-8bit | quantized | -8bit | 70.6B | XL | llama3.1 | mlx-community | 1 | 8.2k |
| backyardai/Llama-3.1-Nemotron-70B-Instruct-GGUF | quantized | GGUF | — | — | llama3.1 | backyardai | 0 | 8k |
| DevQuasar-5/nvidia.Llama-3.1-Nemotron-70B-Instruct-HF-GGUF | quantized | GGUF | — | — | — | DevQuasar-5 | 1 | 5.2k |
| mlx-community/Llama-3.1-Nemotron-70B-Instruct-HF-4bit | quantized | -4bit | 70.6B | XL | llama3.1 | mlx-community | 2 | 4.9k |
| sandbox-ai/Llama-3.1-Tango-70b-bnb_4b | quantized | -bnb | 72.7B | XL | llama3.1 | sandbox-ai | 4 | 3.7k |
| Model-SafeTensors/Llama-3.1-Tango-70b | merge | — | 70.6B | XL | — | Model-SafeTensors | 0 | 3.8k |
| gaianet/Llama-3.1-Nemotron-70B-Instruct-HF-GGUF | quantized | GGUF | — | — | llama3.1 | gaianet | 0 | 3.3k |
| crestf411/L3.1-nemotron-sunfall-v0.7.0 | finetune | — | 70.6B | XL | llama3 | crestf411 | 19 | 2k |
| unsloth/Llama-3.1-Nemotron-70B-Instruct | finetune | — | 70.6B | XL | llama3.1 | unsloth | 6 | 2.2k |
| xmadai/Llama-3.1-Nemotron-70B-Instruct-xMADai-INT4 | quantized | INT4 | — | — | llama3.1 | xmadai | 4 | 1.8k |
| nitky/Llama-3.3-SuperSwallow-70B-Instruct-v0.1 | merge | — | 70.6B | XL | llama3.1gemma | nitky | 2 | 1.6k |
| mlx-community/Llama-3.1-Nemotron-70B-Instruct-HF | finetune | — | 70.6B | XL | llama3.1 | mlx-community | 3 | 1.4k |
| ssmits/Llama-3.1-Nemotron-92B-Instruct-HF-late | finetune | — | 91.9B | XL | — | ssmits | 2 | 1.3k |
| Ppoyaa/MythoNemo-L3.1-70B-v1.0 | finetune | — | 70.6B | XL | llama3.1 | Ppoyaa | 5 | 1.1k |
| KaraKaraWitch/GoldDiamondGold-L33-70b | merge | — | 70.6B | XL | — | KaraKaraWitch | 6 | 1k |
| mysticbeing/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-DYNAMIC | finetune | -FP8 | 70.6B | XL | llama3.1 | mysticbeing | 3 | 1.1k |
| ssmits/Llama-3.1-Nemotron-92B-Instruct-HF-early | finetune | — | 91.9B | XL | — | ssmits | 2 | 965 |
| OpenBuddy/openbuddy-nemotron-70b-v23.1-131k | finetune | — | 70.6B | XL | llama3.1 | OpenBuddy | 4 | 860 |
| sandbox-ai/Llama-3.1-Tango-70b | finetune | — | — | — | llama3.1 | sandbox-ai | 8 | 516 |
| featherless-ai-quants/nvidia-Llama-3.1-Nemotron-70B-Instruct-HF-GGUF | quantized | GGUF | — | — | — | featherless-ai-quants | 0 | 849 |
| sequelbox/Llama3.1-70B-PlumChat | merge | — | 70.6B | XL | llama3.1 | sequelbox | 0 | 825 |
| olaverse/MIST-1-70B | merge | — | 70.6B | XL | llama3.1 | olaverse | 1 | 603 |
| VPTQ-community/Llama-3.1-Nemotron-70B-Instruct-HF-v8-k65536-65536-woft | quantized | — | 11.3B | L | llama3.1 | VPTQ-community | 5 | 336 |
| win28703/Llama-3.1-Nemotron-70B-Instruct-HF-Q8-mlx | quantized | mlx | 70.6B | XL | llama3.1 | win28703 | 0 | 451 |
| RohitPoreddy/Llama-3.1-Nemotron-70B-Instruct-HF-Q4-mlx | quantized | mlx | 70.6B | XL | llama3.1 | RohitPoreddy | 0 | 402 |
| nonetrix/llama-3.1-70B-nemotron-lumiyale-120B | merge | — | 119B | XXL | — | nonetrix | 2 | 290 |
| cnfusion/Llama-3.1-Nemotron-70B-Instruct-HF-Q2-mlx | quantized | mlx | 70.6B | XL | llama3.1 | cnfusion | 0 | 313 |
| VPTQ-community/Llama-3.1-Nemotron-70B-Instruct-HF-v16-k65536-256-woft | quantized | — | 5.91B | M | llama3.1 | VPTQ-community | 1 | 258 |
| Date | Commit | SHA |
|---|---|---|
| 2025-04-13 | Update README.md | 031d404 |
| 2024-10-25 | Update README.md | fac73d3 |
| 2024-10-16 | Update README.md | b919e5d |
| 2024-10-16 | Update README.md | dfe497a |
| 2024-10-15 | Update README.md | 250db5c |
| 2024-10-12 | Update README.md | 1bfaa7a |
| 2024-10-12 | Update README.md | 2b144fb |
| 2024-10-12 | Upload folder using huggingface_hub | 475e82d |
| 2024-10-12 | initial commit | 1469505 |
Hosted endpoints observed for Llama-3.1-Nemotron-70B-Instruct-HF, sorted by input price (USD per million tokens). Each row names its observer and date; prices change daily, so check the source before quoting.
| Provider | Model ref | Context | Quant | $/M in | $/M out | tok/s | TTFT | Tools | Status | Observed |
|---|---|---|---|---|---|---|---|---|---|---|
| Featherless hf | nvidia/Llama-3.1-Nemotron-70B-Instruct-HF | — | — | — | — | — | — | — | live | 2026-10-25 |
Ranked by source signals (HN, Lobsters, authority domains, official posts) and recency. Summaries are generated from a stored copy of the article text; every sentence cites a span of that text, and nothing is shown that failed the citation check.
arXiv links found in the model card and its tags. "Peer-reviewed" is shown only with evidence: a published venue in OpenAlex (measured) or the authors' own journal reference on arXiv (reported).
Hub snapshot: 2026-10-25 · HF API: 2026-10-24 21:37 UTC · Providers: 2026-10-24 21:44 UTC · Articles: 2026-10-24 21:45 UTC · Summary: 2026-10-07 15:05 UTC · Last verified 2026-10-25
Email alerts are not open yet; the feeds above are the same data. See feeds for the item schema.
